Real-time data visualization method and device and electronic equipment

Abstract

The invention relates to the technical field of data visualization, in particular to a real-time data visualization method and device and electronic equipment. According to the method, a visual model is constructed based on geometric feature sampling feature points of a stress area of a target product, accurate association is established by combining tactile sensor data and reference points, and flexible binding of vertexes and sensor data is realized by using an index array and a weight array. The real-time stress data is converted into a visual visualization effect through dynamic rendering, the visual visualization effect comprises controlling the displacement of a vertex in the normal direction, actual stress distribution is fitted, the accuracy and physical authenticity of stress visualization are ensured, and hardware performance is brought into full play through programmable logic to improve the real-time rendering efficiency. Besides, through dynamic weight configuration, color mapping texture and other data and model decoupling design, the flexibility and expansibility of the system are enhanced, the cost for adapting to different products or modifying display rules can be reduced, and visual visualization of real-time data of the sensor is comprehensively achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data visualization, and in particular to a real-time data visualization method and device and electronic equipment. BACKGROUND

[0002] Sensor real-time visualization technology is a core supporting technology in the fields of Internet of Things, Industry 4.0, etc., which can visually and real-timely present sensor collected data, and improve data understanding efficiency and decision accuracy. However, the rendering mode of the existing system is usually based on pixel mapping of screen coordinates, directly corresponding sensor data to the screen according to pixel positions, which lacks flexible processing of data. When it is necessary to realize slightly complex display transformations such as eccentric display or scaling, it is necessary to modify the coordinate mapping code or handle extremely complex coordinate mapping relationships, which is low in calculation efficiency and limited in flexibility, or a fixed function pipeline rendering mode is adopted, which depends on the fixed rendering process preset by the GPU hardware and can only complete simple graphics rendering. For example, fixed color or shape rendering cannot take full advantage of the parallel computing advantage of the GPU, lacks dynamic configuration and expansion capability, and is poor in rendering efficiency and effect. SUMMARY

[0003] The embodiments of the present application mainly solve the technical problem of low data rendering efficiency in the related art.

[0004] To solve the above technical problems, one technical solution adopted by the embodiments of the present application is to provide a real-time data visualization method, comprising: sampling a plurality of feature points as vertices of a visualization model according to the geometric features of a stress region of a target product, wherein the feature points are points representing the geometric features of the stress region; determining and labeling the associated attributes of each vertex corresponding to the touch sensor, so as to output the visualization model based on the vertices; obtaining real-time sensor data of the target product; dynamically rendering the real-time sensor data based on the visualization model, so as to output a stress image of the target product.

[0005] In some embodiments, the determination and labeling of the associated attributes of each vertex corresponding to the touch sensor, so as to output the visualization model based on the vertices, comprises: generating a point set of the visualization model by acquiring all the feature points, wherein the point set contains the identification and three-dimensional space coordinates of the feature points; selecting a stress point in the point set as a reference point according to a physical sensing unit of the touch sensor, wherein the stress point corresponds to the physical sensing unit one by one; determine an index array and a weight array corresponding to the vertex according to the neighboring reference points of the vertex; label an associated attribute of the vertex according to the index array and the weight array, so as to output a visual model corresponding to a target product based on the vertex.

[0006] In some embodiments, the determining the index array and the weight array corresponding to the vertex according to the neighboring reference points of the vertex comprises: acquiring a preset number of reference points with the shortest distance from the vertex as the neighboring reference points of the vertex; acquiring indexes of the neighboring reference points in the point set to constitute the index array corresponding to the vertex; calculating a distance from the vertex to each of the neighboring reference points and a corresponding distance weight; determining a normalized weight of each of the neighboring reference points according to the distance weight; determining the weight array corresponding to the vertex based on the normalized weights of the preset number of neighboring reference points.

[0007] In some embodiments, the number of the neighboring reference points corresponding to each of the vertices is greater than three.

[0008] In some embodiments, the dynamic rendering of the real-time sensor data based on the visual model comprises: parsing the real-time sensor data to acquire a force condition of a force point, the force condition comprising a force vector array; determining a force vector borne by a corresponding vertex according to the force vector array of the force point; determining a movement trajectory of the vertex according to the force vector, and indicating the force condition of the vertex in the form of a visual marker according to the force vector.

[0009] In some embodiments, the determining the movement trajectory of the vertex according to the force vector comprises: acquiring a component of the force vector in a normal direction of the vertex as a displacement amount; controlling the vertex to move along the normal direction according to the displacement amount.

[0010] In some embodiments, the visual marker is an indication arrow, and the indicating the force condition of the vertex in the form of the indication arrow according to the force vector comprises: determining a direction of the indication arrow according to a direction of the force vector; determining a length of the indication arrow according to a size of the force vector; acquiring position information of the vertex as a starting point of the arrow, and generating the indication arrow in combination with the direction and the length.

[0011] In some embodiments, the analyzing the real-time sensor data further comprises obtaining a color mapping texture, the color mapping texture being used to define a mapping relationship between force value size and color; and the indicating the force condition of the vertex according to the force vector in the form of an indication arrow further comprises: determining a color of the indication arrow according to the force value size of the force vector and the color mapping texture.

[0012] To solve the above technical problems, another technical scheme adopted by the embodiments of the present application is to provide a real-time data visualization device, comprising: a sampling module configured to sample a plurality of feature points as vertices of a visualization model according to geometric features of a force-receiving area of a target product, wherein the feature points are points representing the geometric features of the force-receiving area; a model output module configured to determine and label an associated attribute corresponding to each of the vertices based on a touch sensor, so as to output the visualization model based on the vertices; a sensor module configured to obtain real-time sensor data of the target product; an image rendering module configured to dynamically render the real-time sensor data based on the visualization model, so as to output a force image of the target product.

[0013] To solve the above technical problems, another technical scheme adopted by the embodiments of the present application is to provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the real-time data visualization method as described above.

[0014] Different from the related art, the embodiment of the present application provides a real-time data visualization method, device and electronic equipment. The method constructs a visualization model based on the sampling of feature points of the target product stress area geometric characteristics, establishes accurate association combined with the reference point and haptic sensor data, realizes flexible binding of the vertex and sensor data by using the index array and weight array, and converts the real-time stress data into intuitive visualization effect through dynamic rendering, including controlling the displacement of the vertex along the normal direction, and intuitively reflecting the stress condition through the indication arrow of different directions / lengths / colors, which not only conforms to the actual stress distribution, ensures the accuracy and physical authenticity of the stress visualization, but also fully plays the hardware performance through programmable logic to improve the real-time rendering efficiency. In addition, through the decoupling design of dynamic weight configuration and color mapping texture and the like data and model, the flexibility and expandability of the system are enhanced, the cost of adapting different products or modifying the display rule can be reduced, and the intuitive visualization of the sensor real-time data is comprehensively realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] One or more embodiments are illustrated by way of example in the figures that are part of this document, and which illustrate the principles of the embodiments. The embodiments are not limited to the examples that are described.

[0016] Figure 1 A flowchart of a real-time data visualization method of some embodiments of the present application; Figure 2 A flowchart of marking associated attributes of some embodiments of the present application; Figure 3 A structural diagram of a real-time data visualization device of some embodiments of the present application; Figure 4 A structural diagram of an electronic equipment of some embodiments of the present application. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments.

[0018] In the present application, the phrase "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment that is not mutually exclusive with other embodiments.

[0019] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise explicitly specified.

[0020] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0021] The technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0022] In a first aspect, the present application provides a real-time data visualization method, please refer to Figure 1 The method comprises: S11, sampling a plurality of feature points as vertices of a visualization model according to the geometric characteristics of the stress region of the target product, wherein the feature points are points representing the geometric characteristics of the stress region.

[0023] In the embodiments of the present application, the target product refers to an object or device whose stress condition is monitored by a sensor, specifically, a real-time visualized object or device whose stress state is monitored by a tactile sensor. For example, an intelligent glove that monitors contact pressure or a car seat that monitors the pressure distribution of human sitting posture, etc. The present scheme provides a real-time visualization method for stress data of the target product based on multiple sensors.

[0024] In the present scheme, the basic model of the target product can be obtained first, and the visualization model of the target product is determined based on the basic model. For example, the CAD / CAE model of product design is obtained as the basic model of the target product, and sampling is performed in the basic model for the key stress region or sensor distribution region to obtain a plurality of feature points as the vertices of the visualization model. The sampling strategy of the feature points can use uniform grid sampling or adaptive sampling algorithm based on curvature to extract a group of points that can fully represent the geometric characteristics of the region as the vertices of the visualization model. In this way, the vertices of the visualization model directly correspond to the key geometric positions of the stress region / sensor distribution region. For example, the gripping surface and fingertips of the intelligent glove, or the human contact range of the car seat, etc. sensitive points of stress. In this way, the redundancy of indiscriminate sampling is avoided, so that the subsequent rendering of the stress image can truly reflect the actual stress condition.

[0025] S12, determine and label the associated attributes of each vertex corresponding to the haptic sensor, so as to output the visualization model based on the vertex. The present scheme labels the associated attributes of each vertex after sampling to the fixed point, such as the vertex corresponding sensor, coordinate, force threshold and the like, so as to accurately bind the sensor data and the vertex of the visualization model, reduce the data disorder, and make the subsequent rendering clear to the force point and force condition.

[0026] In the embodiments of the present application, please combine Figure 2 The above determining and labeling the associated attributes of each vertex corresponding to the haptic sensor, so as to output the visualization model based on the vertex, further comprises: S121, obtain a point set of the visualization model generated by all the feature points, and the point set contains the identification and three-dimensional space coordinates of the feature points. Specifically, a point set V0 containing all point identifications and three-dimensional space coordinates [x, y, z] of the feature points is generated based on the sampling result of the feature points. The point identification is the unique identification of the feature point in the point set, which can be represented by letters or numbers. The three-dimensional space coordinates are the three-dimensional space coordinates of the feature points, which can be directly taken as the coordinate system of the basic model in the actual application scenario.

[0027] S122, select the force points in the point set as reference points according to the physical sensing units of the haptic sensor, and the force points correspond to the physical sensing units one by one. According to the physical layout of the actual sensing unit (Taxel, haptic pixel point) of the haptic sensor, the points corresponding to the physical sensing units of the haptic sensor one by one are determined from the point set V0 as reference points. These reference points are directly bound to the haptic sensor data, such as the force data of a sensing unit of the sensor corresponding to a reference point. This step can be manually selected and labeled by workers, or automatically selected and labeled by algorithms, so that each reference point R[i] can correspond to the i-th force vector F[i]={x, y, z} in the sensor data group F[].

[0028] S123. Determine the index array and weight array corresponding to the vertex based on the neighboring reference points. For each vertex, find the reference points around it as its neighboring reference points. Use the index array to record the identifiers of these neighboring reference points for quick vertex location, and use the weight array to record the influence of each neighboring reference point on the vertex; for example, the closer the distance, the greater the weight. Compared to simple point-by-point mapping, the vertices in this scheme are not directly bound to sensor data, but are indirectly associated through reference points. Utilizing the calculation of the vertex's own force using neighboring reference points can more smoothly reflect the force distribution. For example, using weighted average of the force values ​​of neighboring reference points, especially for the region between sensor detection points, can make the subsequent visualization rendering effect more continuous and realistic. On this basis, when the sensor layout is fine-tuned, such as adding or removing sensing units, only the neighboring reference points and their corresponding weights need to be updated, without reconstructing the entire model, resulting in greater flexibility.

[0029] In some embodiments, determining the index array and weight array corresponding to the vertex based on the vertex's nearest reference points further includes: A predetermined number of reference points with the shortest distance to the vertex are obtained as the vertex's nearest reference points. Based on the point set V0 obtained above, for a vertex P (in this embodiment, vertex P can be any vertex in point set V0 other than the reference points; to distinguish it from the reference points in point set V0, it can also be represented as a non-reference point P), a predetermined number of reference points with the shortest spatial distance to it are found in the reference point set R[] as the vertex P's nearest reference points. These nearest reference points are the reference points that have the greatest impact on the force exerted on vertex P. Specifically, in this embodiment, the K-nearest neighbor algorithm can be used to determine the nearest reference points of vertex P.

[0030] The indices of the nearest reference points in the point set are obtained to form the index array corresponding to the vertex. In this embodiment, for each vertex, four nearest reference points [R[N1], R[N2], R[N3], R[N4]] can be determined, and the index array [N1, N2, N3, N4] corresponding to the vertex is formed based on the indices of these four nearest reference points.

[0031] Calculate the distance from the vertex to each of the neighboring reference points, and the corresponding distance weight. Specifically, calculate the Euclidean distances [D1, D2, D3, D4] from vertex P to its four neighboring reference points. To ensure the sum of the weights is 1, this embodiment uses inverse distance-weighted interpolation to calculate the corresponding distance weights, as shown in the following formula: W i =1 / (D i +ε); where W i D represents the distance weight of the i-th nearest reference point. iis the Euclidean distance between the i-th neighboring reference point and the vertex P, and ε is a very small constant to prevent division by zero error.

[0032] According to the distance weight, a normalized weight of each neighboring reference point is determined, and the calculation formula is as follows: P i =W i / (W1+W2+W3+W4); wherein, P i is the normalized weight of the i-th neighboring reference point, and W1, W2, W3, and W4 are the distance weights of the four neighboring reference points, respectively.

[0033] Based on the normalized weights of the preset number of neighboring reference points, a weight array corresponding to the vertex is determined. Taking the above example of four neighboring reference points, the weight array [P1, P2, P3, P4] corresponding to the vertex P can be determined.

[0034] S124, according to the index array and the weight array, the associated attribute of the vertex is labeled, so as to output the visual model corresponding to the target product based on the vertex. Based on the above calculation method, the index array [N1, N2, N3, N4] and the weight array [P1, P2, P3, P4] obtained can be used as the associated attribute of the vertex P together and stored in the visual model.

[0035] Based on this, the force condition of each vertex can be associated with the corresponding sensor data, and how to calculate the force of the corresponding vertex based on the sensor data. Based on these vertices with associated attributes, the visual model of the target product is constructed, so that each vertex in the model has a rule to associate its virtual force with the four nearest actual sensor data. Therefore, in this scheme, 4 nearest reference points are bound to each vertex, which is equivalent to connecting each vertex of the virtual model to the real-time data of multiple sensors, rather than responding to a single sensor. This multi-reference point fusion method can fully integrate the multi-region and multi-dimensional force information collected by the sensor into the model, improving the richness of the sensor data. Rich sensor data can make the visualization result more accurately present the relevance and distribution characteristics of regional force; moreover, in the scheme, the distance weight and normalization processing make the vertex force and the force value change of the neighboring reference points form a weighted linkage, that is, the closer the reference point (the more direct the sensitivity feedback of the region), the greater the influence on the vertex force. When the sensor senses a small force value fluctuation, the corresponding vertex can quickly respond to this change through the weight, thereby improving the sensitivity. At the same time, the distance weight is designed based on the nearest (several) reference points, which also conforms to the actual physical law, making the vertex force more close to the real force condition in its actual space range, and the accuracy of model transformation is also higher.

[0036] It can be understood that the above four adjacent reference points are only an example of the preset number in the embodiment, and the number can be different from the number in actual application. For example, if the accuracy requirement is high, six adjacent reference points can be set, and for example, if the efficiency requirement is high, three adjacent reference points can be set, and the efficiency is improved by reducing the amount of calculation. The number of settings can take into account the visual effect and performance to determine.

[0037] In actual application, Three.js can be used to create a 3D scene, a camera and a renderer, and the visualization model obtained in the above steps is loaded into the scene, which can be exported in glTF format, not only enabling the model to be efficiently loaded into the scene (such as a rendering environment based on WebGL, Unity, Three.js, etc.), but also retaining the associated attribute information of the vertex, ensuring correct association of sensor data during subsequent dynamic rendering.

[0038] S13, obtaining real-time sensor data of the target product. In actual use, a specific shader material can be created for the visualization model, wherein the material is used to dynamically adjust the position and color of the vertex according to the real-time force data. Before obtaining the real-time sensor data, a WebSocket client can be initialized to establish a connection with a sensor data server, so as to receive the real-time sensor data of the target product.

[0039] S14, dynamically rendering the real-time sensor data based on the visualization model to output a force image of the target product. Based on the visualization model obtained above, the index array therein can be used to quickly determine the reference point data corresponding to the vertex, that is, the adjacent reference point, and the weight array can be used to directly calculate the real-time force condition by weighting, thereby simplifying the real-time force calculation, reducing the amount of calculation during dynamic rendering in actual force, and improving the real-time performance.

[0040] In some embodiments, the above dynamic rendering of the real-time sensor data based on the visualization model comprises: parsing the real-time sensor data to obtain a force condition of a force point, the force condition comprising a force vector array; determining a force vector of a corresponding vertex according to the force vector array of the force point.

[0041] In this embodiment, the sensor data packets from the server can be received in real time through WebSocket, and the force vector array F [{x, y, z},...] is parsed according to the sensor data. Still taking the above four adjacent reference points as an example, for a vertex V in the visualization model, the index array [N1, N2, N3, N4] and the weight array [P1, P2, P3, P4] can be read from the associated attributes of the vertex V, and the force vectors F [N1] to F [N4] of the corresponding four adjacent reference points can be determined from the force vector array F. The force vector P = {Px, Py, Pz} of the vertex V is calculated in parallel: Px = F[N1].x * P1 + F[N2].x * P2 + F[N3].x * P3 + F[N4].x * P4 Py = F[N1].y * P1 + F[N2].y * P2 + F[N3].y * P3 + F[N4].y * P4 Pz = F[N1].z * P1 + F[N2].z * P2 + F[N3].z * P3 + F[N4].z * P4 wherein Px, Py, and Pz correspond to the x-axis, y-axis, and z-axis directions, respectively.

[0042] The movement trajectory of the vertex is determined according to the force vector, and the force condition of the vertex is indicated in the form of a visualization marker according to the force vector. In this scheme, the force condition is displayed in the form of a visualization marker, and the force condition and movement trajectory of the vertex are converted into intuitive visual images, which can intuitively present the force conditions at different positions of the model and facilitate the user to quickly understand the force state and movement trend.

[0043] In some embodiments, the visualization marker is an indication arrow. In this case, the force condition of the vertex is indicated in the form of an indication arrow according to the force vector, including determining the direction of the indication arrow according to the direction of the force vector, determining the length of the indication arrow according to the size of the force vector, obtaining the position information of the vertex as the starting point of the arrow, and generating the indication arrow in combination with the direction and length. Specifically, the position of the vertex V can be taken as the starting point of the indication arrow, the direction of the force vector P obtained by the above interpolation calculation can be taken as the direction of the indication arrow, for example, if the force vector P is directed to the upper left, the indication arrow is directed to the upper left, the length of the indication arrow is determined according to the size (i.e. the module length) of the force vector P, and a three-dimensional arrow is dynamically generated or updated as the indication arrow.

[0044] The method further includes: obtaining a component of the force vector in a normal direction of the vertex as a displacement amount; and controlling movement of the vertex along the normal direction according to the displacement amount. Specifically, a component of the force vector P in the normal direction of the vertex (i.e., a normal force) is taken as the displacement amount, and the vertex is driven to move along the normal direction according to the displacement amount. If the force is outward, the vertex protrudes outward; if the force is inward, the vertex is recessed inward, thereby generating a protrusion or a recess in the image.

[0045] Based on this, the displacement of the vertex in the present scheme depends on the component of the force vector in the normal direction, thereby avoiding redundant calculation in irrelevant directions, while conforming to the physical intuition that the extrusion / stretch of the force on the surface is mainly along the vertical direction, and the direction and length of the arrow are directly generated based on the force vector data, which is simple to calculate and can reduce delay, and is suitable for real-time dynamic rendering.

[0046] It can be understood that the above embodiment is described by taking the arrow as an example of the visual marker, and is not limited thereto. In some other application scenarios, the visual marker can include an arrow, a color block, a geometric deformation body, or other forms. For example, the direction of the force is indicated by an arrow, the size and nature of the force are conveyed by a color block, the deformation of the object after being subjected to the force is displayed by a geometric deformation body, and the like, as well as combinations thereof, all of which belong to the scope of the visual marker in the present scheme.

[0047] In some embodiments, the analyzing the real-time sensor data further includes obtaining a color mapping texture, the color mapping texture being used to define a mapping relationship between a force value and a color; and the indicating, in the form of an arrow, the force condition of the vertex according to the force vector further includes determining a color of the arrow according to the force value of the force vector and the color mapping texture, so that the color of the arrow can be used to map the size of the force. In this embodiment, when the sensor data is analyzed, the color mapping rule can be obtained at the same time, for example, a gradient from blue to red is set for a force value of 0-10, and then a corresponding color is found from the color mapping texture according to the size of the force vector, as the color of the arrow, for example, an arrow indicating a large force value is red, and an arrow indicating a small force value is blue, thereby further strengthening the contrast between the force values through the color, so as to facilitate the user to quickly distinguish the size of the force in the vision.

[0048] Based on this, the scheme constructs a visual model by sampling feature points based on the geometric characteristics of the stress area of the target product, establishes accurate association combining tactile sensor data and reference points, realizes flexible binding of vertices and sensor data by using index array and weight array, and converts real-time stress data into intuitive visual effects through dynamic rendering, including controlling the displacement of the control vertices along the normal direction, and intuitively reflecting the stress situation through the indication arrows of different directions / lengths / colors, which not only conforms to the actual stress distribution, ensures the accuracy and physical reality of stress visualization, but also fully plays the hardware performance through programmable logic to improve the real-time rendering efficiency. In addition, through the decoupling design of dynamic weight configuration and color mapping texture and the like data and model, the flexibility and expandability of the system are enhanced, the cost of adapting different products or modifying display rules can be reduced, and the intuitive visualization of real-time sensor data is comprehensively realized.

[0049] The embodiment of the application provides a real-time data visualization device, please combine Figure 3 The real-time data visualization device 200 includes a sampling module 201, a model output module 202, a sensor module 203 and an image rendering module 204. Specifically, the sampling module 201 can sample a plurality of feature points as vertices of a visual model according to the geometric characteristics of the stress area of the target product, wherein the feature points are points representing the geometric characteristics of the stress area; the model output module 202 can determine and label the associated attributes corresponding to each vertex based on the tactile sensor, so as to output the visual model based on the vertices; the sensor module 203 can obtain real-time sensor data of the target product; and the image rendering module 204 can dynamically render the real-time sensor data based on the visual model to output a stress image of the target product.

[0050] It should be noted that the above-mentioned real-time data visualization device can execute the real-time data visualization method provided by the embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method. Technical details not described in detail in the real-time data visualization device embodiment can be referred to the real-time data visualization method provided by the embodiment of the application.

[0051] Please refer to Figure 4 , Figure 4 is a hardware structure schematic diagram of the electronic device 300 provided by the embodiment of the application. The electronic device 300 includes one or more processors 301 and a memory 302, Figure 4 In the embodiment, the processor 301 is taken as an example. The processor 301 and the memory 302 can be connected through a bus or other means, Figure 4 In the embodiment, the connection through the bus is taken as an example.

[0052] The memory 301, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the real-time data visualization method in the embodiment of the application. The processor 301 executes various function applications and data processing of the electronic device 300 by running the non-volatile software programs, non-volatile computer executable programs and modules stored in the memory 302, that is, implements the real-time data visualization method in the above-mentioned method embodiment.

[0053] The memory 302 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the real-time data visualization device, etc. In addition, the memory 302 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 302 can optionally include a memory remotely arranged with respect to the processor 301, which can be connected to the real-time data visualization device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0054] The one or more modules are stored in the memory 302, and when executed by the one or more processors 301, the real-time data visualization method in the above-mentioned method embodiment is executed.

[0055] The above-mentioned product can execute the real-time data visualization method provided by the embodiment of the application, and has the function modules and beneficial effects corresponding to the real-time data visualization method. Technical details not described in detail in the embodiment can be referred to the real-time data visualization method provided by the embodiment of the application.

[0056] The electronic device 300 of the embodiment of the application can exist in various forms, including but not limited to a server, a server cluster, a cloud server, and other electronic devices with data interaction function.

[0057] The embodiment of the application further provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors, so that the above-mentioned one or more processors can execute the real-time data visualization method in any method embodiment.

[0058] The embodiment of the present application further provides a computer program product, which comprises a computer program stored on a nonvolatile computer readable storage medium, and the computer program comprises program instructions, which, when executed by the electronic device, cause the electronic device to perform the real-time data visualization method in the above embodiment.

[0059] The device or equipment embodiments described above are merely illustrative, wherein the unit modules described as separate components may or may not be physically separated, and the components displayed as module units may or may not be physical units, i.e., may be located in one place or distributed on multiple network module units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0061] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in detail; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for visualizing real-time data, characterized in that, include: Multiple feature points are sampled as vertices of the visualization model based on the geometric features of the stress area of ​​the target product, wherein the feature points are points representing the geometric features of the stress area; The associated attributes corresponding to each vertex are determined and labeled based on the tactile sensor, so as to output the visualization model based on the vertex; Acquire real-time sensor data of the target product; The real-time sensor data is dynamically rendered based on the visualization model to output a force image of the target product.

2. The method according to claim 1, characterized in that, The step of determining and labeling the associated attributes corresponding to each vertex based on the tactile sensor, and outputting the visualization model based on the vertices, includes: Obtain a point set for a visualization model from all the aforementioned feature points, the point set including the identifiers and three-dimensional spatial coordinates of the feature points; The force points in the set of points are selected as reference points according to the physical sensing unit of the tactile sensor, and the force points correspond one-to-one with the physical sensing unit; The index array and weight array corresponding to the vertex are determined based on the nearest reference points of the vertex; The associated attributes of the vertices are labeled according to the index array and the weight array, so as to output a visualization model corresponding to the target product based on the vertices.

3. The method according to claim 2, characterized in that, The step of determining the index array and weight array corresponding to the vertex based on the neighboring reference points of the vertex includes: Obtain a preset number of reference points that are closest to the vertex as the vertex's nearest reference points; Obtain the indices of the nearest reference points in the point set, and form an index array corresponding to the vertex; Calculate the distance from the vertex to each of the neighboring reference points, and the corresponding distance weight; Based on the distance weights, determine the normalized weights for each neighboring reference point; Based on the normalized weights of a preset number of nearby reference points, the weight array corresponding to the vertex is determined.

4. The method according to claim 3, characterized in that, The number of neighboring reference points corresponding to each vertex is greater than three.

5. The method according to claim 3, characterized in that, The dynamic rendering of the real-time sensor data based on the visualization model includes: The real-time sensor data is analyzed to obtain the force situation at the force point, and the force situation includes a force vector array; The force vector of the corresponding vertex is determined based on the force vector array of the force points; The movement trajectory of the vertex is determined based on the force vector, and the force situation of the vertex is indicated by the force vector in the form of visual markers.

6. The method according to claim 5, characterized in that, Determining the movement trajectory of the vertex based on the force vector includes: The component of the force vector in the normal direction at the vertex is obtained as the displacement. The vertex is moved along the normal direction based on the displacement.

7. The method according to claim 5, characterized in that, The visual marker is an indicator arrow, which, in the form of the indicator arrow, indicates the force situation of the vertex according to the force vector, including: The direction of the indicator arrow is determined based on the direction of the force vector; The length of the indicator arrow is determined based on the magnitude of the force vector; The position information of the vertex is obtained as the starting point of the arrow, and the indicator arrow is generated by combining the direction and length.

8. The method according to claim 6, characterized in that, The parsing of the real-time sensor data further includes acquiring a color mapping texture, which is used to define the mapping relationship between force magnitude and color; the step of indicating the force on the vertex in the form of an indicator arrow based on the force vector also includes: The color of the indicator arrow is determined based on the magnitude of the force vector and the color mapping texture.

9. A real-time data visualization device, characterized in that, include: The sampling module is used to sample multiple feature points as vertices of the visualization model based on the geometric features of the stress area of ​​the target product, wherein the feature points are points representing the geometric features of the stress area; The model output module is used to determine and label the associated attributes corresponding to each vertex based on the tactile sensor, so as to output the visualization model based on the vertex; The sensor module is used to acquire real-time sensor data of the target product; The image rendering module is used to dynamically render the real-time sensor data based on the visualization model to output a force image of the target product.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the real-time data visualization method as described in any one of claims 1-8.

Images