Intelligent product simulation test scene construction method based on virtual reality

Abstract

The invention discloses an intelligent product simulation test scene construction method based on virtual reality, and the method comprises the steps: obtaining intelligent product parameters, and determining test scene elements; obtaining a mathematical twinborn model of the intelligent product and dynamic estimation of scene elements; mapping the dynamic estimation of the scene element to a mathematical twinborn model of the intelligent product to obtain a mathematical twinborn model of the test scene; performing simulation test operation on the mathematical twinborn model of the test scene based on the test model, and feeding back two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twinborn model for updating to obtain a two-dimensional virtual point; performing back projection based on the two-dimensional virtual point to recover a virtual point cloud; and performing three-dimensional modeling according to the virtual point cloud to form a virtual simulation test scene of the intelligent product. The method not only can accurately simulate test scene elements, but also has good interpretability, and can be directly applied to a virtual reality-based intelligent product simulation test scene construction system.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a method for constructing a simulation test scenario of an intelligent product based on virtual reality. Background Art

[0002] By using virtual reality to simulate test scenarios, the test cost can be reduced and the test cycle can be shortened. Using virtual reality technology, various complex test scenarios can be easily constructed to comprehensively test the performance of smart products in different situations.

[0003] The traditional method of constructing test scenarios for smart products mainly relies on the construction of actual physical environments and physical testing, but this method usually requires a lot of time, manpower and material resources.

[0004] In contrast, the use of virtual reality to simulate test scenarios has obvious advantages. It can quickly build various complex test scenarios in a virtual environment without the need for actual physical construction, greatly reducing the cost of testing. At the same time, virtual reality technology can adjust the test scenario at any time as needed, shortening the test cycle. In addition, through precise parameter settings and simulation algorithms, the performance of smart products in different situations can be tested more comprehensively and accurately, improving the reliability and effectiveness of the test. Summary of the invention

[0005] The purpose of the present invention is to provide a method for constructing a simulation test scenario of an intelligent product based on virtual reality.

[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0007] The first aspect of the present invention provides a method for constructing a virtual reality-based intelligent product simulation test scenario, comprising the following steps:

[0008] S100 obtains smart product parameters and a test purpose, and determines test scenario elements based on the test purpose;

[0009] S200 performs simulation based on the smart product parameters to obtain a mathematical twin model of the smart product; performs data dynamic evolution on the test scene elements to obtain a dynamic estimation of the test scene elements;

[0010] S300 maps the dynamic estimation of the test scene elements to the mathematical twin model of the smart product to obtain the mathematical twin model of the test scene;

[0011] S400 constructs a test model based on the test purpose, simulates a test operation on the mathematical twin model of the test scene based on the test model, feeds back the two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating, and obtains a two-dimensional virtual point;

[0012] S500 back-projects the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud;

[0013] S600 performs three-dimensional modeling according to the virtual point cloud to form a virtual simulation test scene for the smart product.

[0014] As a further method, the method for performing simulation based on the smart product parameters includes:

[0015] Create a geometric model of the smart product using a 3D modeling tool based on the collected smart product parameters, and assign material properties to the geometric model;

[0016] In the geometric model, the performance parameter equations of the smart product are distributed by the finite element method, and the differential partial equations are combined. The differential partial equations are solved by the conjugate gradient method to obtain the mathematical twin model of the smart product.

[0017] As a further method, the method for dynamically evolving data of the test scenario elements includes:

[0018] Set a time window, collect the real-time status of the test scene elements based on the time window, and obtain the time series status data of the test scene elements;

[0019] Based on the convolutional neural network, the state characteristics of the test scene elements are learned from the time series state data. The convolution operation is performed on the time series state data in the time dimension through the convolution kernel. The expression is:

[0020]

[0021] Among them, y l [n] is the output of the lth convolutional layer at time n, M l is the number of convolution kernels in the lth layer, J l is the number of sequences that make up the lth convolutional layer, x l [nk] is the input sequence of the l-th convolutional layer at time nk, gl,m,j[i] is the weight value of the j-th sequence of the m-th convolution kernel in the l-th convolutional layer at time i, and fl,m,j[i+k] is the activation value of the j-th sequence of the m-th convolution kernel in the l-th convolutional layer at time i+k;

[0022] The state of the convolutional neural network model is updated by the recurrent neural network based on the learned state features of the test scene elements to perform dynamic data evolution and obtain a dynamic estimate of the test scene elements.

[0023] As a further method, the method of mapping the dynamic estimation of the test scene element to the mathematical twin model of the smart product includes:

[0024] The dynamic estimation of the test scene elements is time-aligned with the mathematical twin model of the smart product through the dynamic time warping algorithm;

[0025] Each time is mapped by constructing a mapping function, where the expression of the mapping function is:

[0026] F(O)(Rp+t,S(sp,0),TS(TM,U(Rp+t),V(R,+t)))

[0027] Among them, F(O) is the output of the mapping function, O is the input object, Rp is the position vector, t is the time offset, that is, the change of Rp over time, S(sp,θ) is the position and direction feature function, sp is the spatial position, θ represents the direction change, TS represents the spatial model scale, TM represents the time model scale, U(Rp+t) represents the correlation function between the position vector and the time offset, and V(Rp+t) represents the transfer matrix between the position vector and the time offset.

[0028] As a further method, the method for constructing a test model based on the test purpose includes:

[0029] Obtain directly relevant input variables based on the test purpose, and determine the value range of the input variables based on the test purpose and the design specifications of the smart product;

[0030] For test purposes that can be quantified numerically, determine the quantitative output as the output of the test model. When the test purpose involves classification and / or state judgment, determine the qualitative output as the output of the test model.

[0031] Construct a test function based on the test purpose, and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and a test function combination to construct a test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model combined with a test function; if the purpose of the test involves state judgment, use a neural network algorithm and a test function combination to build a test model.

[0032] As a further method, the method of simulating the test operation on the mathematical twin model of the test scene based on the test model, and feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating includes:

[0033] The input parameters of the test model and the current state parameters of the mathematical twin model are passed to the test model, and the mathematical twin model is simulated and tested through the test model;

[0034] Extract 2D and 1D feature information from simulation test results, including the geometric boundaries and motion trajectories of smart products;

[0035] Feeding the processed two-dimensional feature information and one-dimensional feature information back to the mathematical twin model of the test scene, adjusting the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and the one-dimensional feature information, and updating the state of the model;

[0036] Using the updated mathematical twin model, a two-dimensional virtual point is generated in the two-dimensional space, and the expression is:

[0037]

[0038] Among them, f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane respectively, σ x and σ y They represent the dispersion of the two-dimensional virtual points in the x direction and the y direction respectively. ρ is the correlation coefficient between the x direction and the y direction, which means the linear correlation between the two coordinates. Its value is between -1 and 1, where 1 represents a completely positive correlation, -1 represents a completely negative correlation, and 0 represents no correlation.

[0039] As a further method, the method of back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore the virtual point cloud includes:

[0040] The contour features of two-dimensional virtual points are extracted through Moore field tracking algorithm;

[0041] The maximum pooling operation is performed on the contour features through convolution, and the expression is:

[0042]

[0043] Where P(x,y) is the result of the maximum pooling operation at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane, ω is the size of the window in the maximum pooling operation, i and j are the indices of the pixel positions in the window in the maximum pooling operation, and max i∈[x,x+ω-1]max j∈[y,y+ω-1] Indicates that within the maximum pooling window, for each position (x, y), find the maximum value of all positions in the window, a and b are half of the size of the convolution kernel in the horizontal and vertical directions, respectively, m and n are the indexes of the convolution kernel position in the convolution operation, I(x-i+m,y-j+n) represents the eigenvalue of the current position (x, y) combined with the offset of the convolution kernel, and K(m,n) is the weight value of the convolution kernel at position (m, n);

[0044] Back-project the contour features, the expression is:

[0045] (u,v)→K -1 [u,v,1] T →[R|t] -1 ·(K -1 [u,v,1] T )·z

[0046] Among them, (u, v) is the pixel coordinates of the contour feature for the maximum pooling operation, → represents the conversion process, K -1 is the inverse of the internal parameter matrix, [u,v,1] T is the homogeneous coordinates of a two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel;

[0047] The three-dimensional points are obtained based on the contour features after back-projection, and the obtained three-dimensional points are gathered together to form a virtual point cloud.

[0048] As a further method, the method for performing three-dimensional modeling based on the virtual point cloud includes:

[0049] Cluster analysis is performed on the virtual point cloud, where the objective function of the cluster analysis is expressed as:

[0050]

[0051] Among them, K is the total number of clusters, p represents the points in the virtual point cloud, and P i Represented as the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i ) is the shape regularization term of clustering, α and β are weight parameters for adjusting density and shape influence respectively;

[0052] The Delaunay algorithm is used to triangulate each cluster to obtain a triangular mesh, where the triangulation constraint condition is expressed as:

[0053]

[0054] Among them, t represents a triangular patch, {p i ,p j ,p k} is expressed as the coordinates of three-dimensional space, T i is the set of all triangles in the i-th cluster, R t is the distance threshold, p represents a point in the virtual point cloud, P i Represented as the set of points in the i-th cluster, P i \{p i ,p j ,p k} means to exclude the three points that constitute the triangle patch t, ​​O t is the circumcenter of triangle patch t, ​​R t is the radius of the circumscribed circle;

[0055] The Laplace operator is introduced based on the triangulation result to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression of the solution process is:

[0056]

[0057] Among them, u represents the function to be solved, Ω is the convex hull area of ​​point p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, f is the smoothing constraint function, It is represented as the boundary of region Ω, and u0 is the boundary value;

[0058] All triangular meshes are merged to obtain a virtual simulation test scene.

[0059] In a second aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the method steps described in the first aspect.

[0060] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method steps described in the first aspect.

[0061] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0062] (1) Starting from obtaining smart product parameters and test purposes, the present invention can accurately determine the test scenario elements based on the test purpose and closely link the product with the scenario. This overall consideration can more comprehensively evaluate the scenario conditions of the product in actual use.

[0063] (2) The present invention dynamically evolves the data of test scenario elements and maps them to the mathematical twin model of the smart product, thereby more realistically reflecting the performance of the product in a complex and changing environment.

[0064] (3) The present invention performs back-projection on two-dimensional virtual points to restore virtual point clouds, and then performs three-dimensional modeling based on the virtual point clouds to form a virtual simulation test scene. The two-dimensional virtual points can be used as an intermediate representation of the three-dimensional model, which helps to extract useful two-dimensional feature information from complex three-dimensional structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flowchart of the steps of the method for constructing a virtual reality-based intelligent product simulation test scenario of the present invention.

[0066] Figure 2 It is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0068] Reference Figure 1 As shown, the present invention provides a method for constructing a simulation test scenario of an intelligent product based on virtual reality, comprising:

[0069] S100 obtains smart product parameters and a test purpose, and determines test scenario elements based on the test purpose;

[0070] It should be explained that the parameter data of smart products is a quantitative description of product characteristics. These data can be obtained from product manuals, technical specification documents, data interfaces provided by manufacturers or professional testing equipment. The purpose of the test is to be the starting point and end point of smart product testing, and to clarify the test objectives and expected results. The test objectives can be determined according to different needs and application scenarios, such as: (1) Product performance testing, to test the performance of smart products under various conditions, such as processing speed, response time, and battery life. For example, to test the processor performance of a smart tablet, you can run various applications and games to measure its running speed and smoothness; (2) Reliability testing: Evaluate the reliability and stability of smart products during long-term use. You can simulate different usage environments and working conditions and conduct long-term tests to observe whether the product fails or performance deteriorates;

[0071] It should be understood that test scenario elements refer to various factors that need to be considered during the test process. Together with the product to be tested, they form a complete test scenario. Depending on the purpose of the test, the test scenario elements will also vary;

[0072] In the actual evaluation, the parameter data of the smart bracelet is obtained, including:

[0073] Physical parameters: The main size of the bracelet is 40mm long, 30mm wide, and 10mm thick, the screen diameter is 20mm, the strap length is 200mm, the width is 15mm, the thickness is 2mm, and the weight is 30g;

[0074] Function parameters: built-in acceleration sensor, heart rate sensor, Bluetooth communication module. Acceleration sensor range is ±8g (g is gravity acceleration, g = 9.8m / s 2 ), the heart rate sensor has a measurement range of 30-220 beats / minute, and the Bluetooth communication distance is within 10m. The battery capacity is 100mAh, and the battery life is 7 days in normal use mode;

[0075] The purpose of this test is to test the accuracy and stability of the health monitoring function (heart rate, exercise steps) of the smart bracelet in different exercise states and environments, as well as the stability of the Bluetooth connection;

[0076] In the actual assessment, determine the test scenario elements, including:

[0077] Related to exercise status: Still: the body is still, arms hanging naturally; Slow walking: the speed is about 3-4km / h, arms swing normally; Fast walking: the speed is about 5-6km / h, arms swing with a large amplitude; Running: the speed is about 8-10km / h, arms swing quickly and rhythmically;

[0078] Environment-related: Indoor environment: temperature is 22-25℃, relative humidity is 40-60%, and there is no electromagnetic interference source (such as far away from microwave ovens, wireless routers and other strong interference devices); Outdoor environment: temperature is 15-30℃, relative humidity is 30-70%, and there is a certain amount of electromagnetic interference (such as close to base stations, public Wi-Fi hotspots, etc.);

[0079] S200 performs simulation based on the smart product parameters to obtain a mathematical twin model of the smart product; performs data dynamic evolution on the test scene elements to obtain a dynamic estimation of the test scene elements;

[0080] It should be explained that the dynamic estimation of scene elements refers to the quantitative description of the changes over time of various elements required to build the test scene (such as environmental elements, external interference, user behavior, etc.). For example, when testing an intelligent drone, scene elements may include wind speed, wind direction, air density (environmental factors), interference from other flying objects around (external interference), and the operator's flight instructions (user behavior). These elements are not static, but constantly changing. Their dynamic estimation is to determine the values ​​or change patterns of these elements at different times or in different states;

[0081] It should be explained that in this way, the dynamic changes of the test scene in the time dimension can be simulated, so as to obtain the dynamic estimation of the scene elements. The purpose of this is to more realistically simulate the dynamic changes of various scene elements in the actual use environment. Smart products often face various dynamically changing environmental conditions in actual use. Through this dynamic estimation, the performance, stability and adaptability of the product in different dynamic scenarios can be better evaluated. For example, it can be observed whether the performance of smart products will be affected when the temperature gradually rises, or whether they can maintain a stable connection and normal functions when the network delay fluctuates;

[0082] In the actual evaluation, SolidWorks software was used to create a 3D geometric model, and material properties were further assigned to the 3D geometric model through the marginal material option. For polycarbonate, its density was set to 1.2 g / cm 3 , elastic modulus is 2.4 GPa, Poisson's ratio is 0.38; for silicone, set the density to 1.1 g / cm 3 , elastic modulus is 0.01GPa, Poisson's ratio is 0.49;

[0083] In actual evaluation, according to the working principle of the heart rate sensor, heart rate monitoring is based on photoplethysmography (PPG). There is a relationship between the PPG signal y(t) and the heart rate HR: HR = f(y(t)), where f is a function based on the signal processing algorithm. The heart rate is calculated by filtering the PPG signal, performing peak detection and other operations. Where T is the time interval between two adjacent PPG signal peaks (unit: second); for step monitoring, the acceleration data on three axes (x, y, z) is obtained according to the data of the acceleration sensor as a x (t), a y (t), a z (t), resultant acceleration When a(t) exceeds the threshold of 0.5g and meets certain time interval and waveform characteristics, it is determined as one step; for Bluetooth connection stability, according to the Bluetooth signal transmission model, the relationship between signal strength P(d) and transmission distance d is: Where P0 is the signal strength at the reference distance d0, and n is the path loss index (n=2 indoors and n=3 outdoors). The equation is discretized using the finite element method, and then the conjugate gradient method is used to solve the change of temperature distribution over time and space, thereby constructing a mathematical twin model of the smart bracelet.

[0084] In the actual evaluation, the dynamic evolution of the data in the motion state is as follows: Slow walking scene: Arm swing angle: The arm swing angle θ(t) in the horizontal direction changes with time t as a sine function, (angle unit is degree, t unit is second, one cycle is 4 seconds); acceleration data: according to the principle of kinematics, when walking slowly, the acceleration a along the direction of arm swing is arm (t) will have periodic changes, a arm (t) = 0.2gsin(πt); Fast walking scene: arm swing angle The swing amplitude is larger than slow walking; acceleration data: a arm (t) = 0.3gsin(πt); Running scene: arm swing angle Acceleration data: a arm (t) = 0.5gsin(πt);

[0085] In the actual evaluation, the indoor ambient temperature changes: indoor temperature T indoor (t) There are small fluctuations within a day. (t is in hours); Outdoor ambient temperature change: The fluctuation amplitude is larger than that of indoors; Changes in electromagnetic interference intensity (outdoor scenario): The electromagnetic interference intensity I(t) is related to the signal transmission cycle of nearby base stations. Where I0 is the average interference intensity;

[0086] S300 maps the dynamic estimation of the test scene elements to the mathematical twin model of the smart product to obtain the mathematical twin model of the test scene;

[0087] It needs to be explained that mapping the dynamic estimation of scene elements to the mathematical twin model of smart products is to establish the relationship between scene elements and smart products. In the example of smart drones, mapping the dynamic estimation of environmental factors such as wind speed and wind direction to the mathematical twin model of drones means considering the influence of these environmental factors in the dynamic equations of drones. For example, wind speed will affect the flight speed and attitude of drones, and wind direction will change the flight direction of drones. These influences can be reflected by adding terms related to wind speed and wind direction to the force balance equation of drones;

[0088] It should be explained that through this mapping process, the mathematical twin model of the test scenario obtained is not only a model of the smart product itself, but also a comprehensive model that includes the impact of the test scenario on the product. This model can be used to simulate the behavior and performance of smart products in various scenarios. Continuing with the example of smart drones, this comprehensive model can be used to simulate the performance indicators of drones such as flight trajectory, energy consumption, stability, etc. under different wind speeds, wind directions, and user operations, thereby providing a strong basis for drone testing, optimized design, and fault prediction;

[0089] In the actual evaluation, in the mathematical twin model of the smart bracelet, for the step count monitoring part, the arm swing angle θ(t) and the acceleration a arm (t) is mapped to the input of the acceleration sensor. According to the installation position and direction of the bracelet, the actual movement acceleration of the arm is converted into the acceleration component in the direction of the coordinate axis of the acceleration sensor through coordinate transformation, thereby affecting the calculation of the number of steps; for heart rate monitoring, changes in movement state will affect the PPG signal. When the arm swings, the PPG signal will be disturbed due to changes in blood flow and changes in the light propagation path. When the arm swings at an angle of θ(t), the amplitude of the PPG signal will change according to the relationship y(t)=y0(1-0.05sin(θ(t))), where y0 is the amplitude of the PPG signal at rest;

[0090] In the actual evaluation, in the Bluetooth connection stability part, the electromagnetic interference intensity I(t) in the outdoor environment is mapped to the signal transmission model of the Bluetooth communication module. According to the model of the impact of electromagnetic interference on Bluetooth signals, the interference intensity I(t) will cause the noise of the Bluetooth signal to increase, thereby changing the calculation of the signal strength P(d). The signal strength becomes Where k is the interference coefficient; for temperature changes, the indoor and outdoor ambient temperatures T indoor (t) and T outdoor (t) Mapped into the heat conduction model of the smart bracelet;

[0091] S400 constructs a test model based on the test purpose, simulates a test operation on the mathematical twin model of the test scene based on the test model, feeds back the two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating, and obtains a two-dimensional virtual point;

[0092] It should be explained that different test purposes determine the elements and features that need to be presented in the virtual scene. For example, if the test purpose is to evaluate the collaborative efficiency of robots in a smart factory, then when building a virtual scene, it is necessary to accurately depict the layout of the factory, the movement trajectory of the robots, and the way they interact with each other. By clarifying the test purpose, the details of the virtual scene can be designed in a targeted manner to make it more in line with actual test needs;

[0093] It needs to be explained that simulating the test operation of the mathematical twin model of the test scene can provide a more realistic dynamic effect for the virtual scene. The mathematical twin model combines the physical laws in the actual scene with the characteristics of the smart product. Through the simulation test operation, the state in the virtual scene can be updated in real time, making the virtual scene closer to the actual situation. For example, in the virtual scene of a smart building, by simulating the mathematical twin model, the energy consumption, temperature, lighting and other parameters of the building can be dynamically adjusted according to different weather conditions, personnel activities and other factors, making the virtual scene more realistic and credible;

[0094] In the actual evaluation, according to the set test scene elements (different motion states and environments), the corresponding parameters are input into the test model. For example, in the slow walking scene, the arm swing angle and acceleration calculated previously are converted into the actual input data of the acceleration sensor, and then the test model is operated to obtain the monitoring results such as the number of steps and heart rate.

[0095] S500 back-projects the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud;

[0096] It should be understood that the purpose of restoring the virtual point cloud is to reconstruct a complete three-dimensional scene or object representation. The point cloud is a collection of a large number of three-dimensional points, which can more comprehensively and accurately describe the test scene in which the smart product is located or the three-dimensional shape of the smart product itself. By restoring the virtual point cloud, further three-dimensional modeling can be performed, the spatial relationship between the smart product and the scene can be analyzed, and a basis for subsequent more accurate performance testing and evaluation can be provided;

[0097] In the actual evaluation, the screen plane of the smart bracelet is a two-dimensional plane. A two-dimensional coordinate system (x, y) is established with the center of the screen as the origin. The position of the bracelet in three-dimensional space can be determined by its wearing position on the human arm. The initial position of the bracelet in three-dimensional space is determined according to the arm position of the bracelet. Then, back-projection calculation is performed to restore the two-dimensional virtual point to the three-dimensional coordinate system to obtain multiple three-dimensional points, which constitute a virtual point cloud.

[0098] S600 performs three-dimensional modeling according to the virtual point cloud to form a virtual simulation test scene for the smart product.

[0099] In the actual evaluation, performance testing is performed in simulated test scenarios;

[0100] Specifically, the initial state of the smart bracelet in the scene is set, such as power level, Bluetooth connection status, etc. In this way, a virtual simulation test scene of the smart bracelet is formed, in which the performance of the smart bracelet under different test conditions can be further observed and analyzed.

[0101] In this embodiment, the method for performing simulation based on the smart product parameters includes:

[0102] Create a geometric model of the smart product using a 3D modeling tool based on the collected smart product parameters, and assign material properties to the geometric model;

[0103] In the geometric model, the performance parameter equations of the smart product are distributed by the finite element method, and the differential partial equations are combined. The differential partial equations are solved by the conjugate gradient method to obtain the mathematical twin model of the smart product.

[0104] In this embodiment, the method for dynamically evolving data of the test scenario elements includes:

[0105] Set a time window, collect the real-time status of the test scene elements based on the time window, and obtain the time series status data of the test scene elements;

[0106] Based on the convolutional neural network, the state characteristics of the test scene elements are learned from the time series state data. The convolution operation is performed on the time series state data in the time dimension through the convolution kernel. The expression is:

[0107]

[0108] Among them, y l [n] is the output of the lth convolutional layer at time n, M l is the number of convolution kernels in the lth layer, J l is the number of sequences that make up the lth convolutional layer, x l[nk] is the input sequence of the l-th convolutional layer at time nk, gl,m,j[i] is the weight value of the j-th sequence of the m-th convolution kernel in the l-th convolutional layer at time i, and fl,m,j[i+k] is the activation value of the j-th sequence of the m-th convolution kernel in the l-th convolutional layer at time i+k;

[0109] The state of the convolutional neural network model is updated by the recurrent neural network based on the learned state features of the test scene elements to perform dynamic data evolution and obtain a dynamic estimate of the test scene elements.

[0110] In this embodiment, the method of mapping the dynamic estimation of the test scene element to the mathematical twin model of the smart product includes:

[0111] The dynamic estimation of the test scene elements is time-aligned with the mathematical twin model of the smart product through the dynamic time warping algorithm;

[0112] Each time is mapped by constructing a mapping function, where the expression of the mapping function is:

[0113] F(0)=(Rp+t,S(sp,0),Ts(TM,U(R,+t),V(R,+t)))

[0114] Among them, F(O) is the output of the mapping function, O is the input object, Rp is the position vector, t is the time offset, that is, the change of Rp over time, S(sp,θ) is the position and direction feature function, sp is the spatial position, θ represents the direction change, TS represents the spatial model scale, TM represents the time model scale, U(Rp+t) represents the correlation function between the position vector and the time offset, V(R p +t) is represented as the transfer matrix of position vector and time offset.

[0115] In this embodiment, the method for constructing a test model based on the test purpose includes:

[0116] Obtain directly relevant input variables based on the test purpose, and determine the value range of the input variables based on the test purpose and the design specifications of the smart product;

[0117] For test purposes that can be quantified numerically, determine the quantitative output as the output of the test model. When the test purpose involves classification and / or state judgment, determine the qualitative output as the output of the test model.

[0118] Construct a test function based on the test purpose, and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and a test function combination to construct a test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model combined with a test function; if the purpose of the test involves state judgment, use a neural network algorithm and a test function combination to build a test model.

[0119] In this embodiment, the method of simulating the test operation on the mathematical twin model of the test scene based on the test model, and feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating includes:

[0120] The input parameters of the test model and the current state parameters of the mathematical twin model are passed to the test model, and the mathematical twin model is simulated and tested through the test model;

[0121] Extract 2D and 1D feature information from simulation test results, including the geometric boundaries and motion trajectories of smart products;

[0122] Feeding the processed two-dimensional feature information and one-dimensional feature information back to the mathematical twin model of the test scene, adjusting the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and the one-dimensional feature information, and updating the state of the model;

[0123] Using the updated mathematical twin model, a two-dimensional virtual point is generated in the two-dimensional space, and the expression is:

[0124]

[0125] Among them, f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane respectively, σ x and σ y They represent the dispersion of the two-dimensional virtual points in the x direction and the y direction respectively. ρ is the correlation coefficient between the x direction and the y direction, which means the linear correlation between the two coordinates. Its value is between -1 and 1, where 1 represents a completely positive correlation, -1 represents a completely negative correlation, and 0 represents no correlation.

[0126] In this embodiment, the method of back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud includes:

[0127] The contour features of two-dimensional virtual points are extracted through Moore field tracking algorithm;

[0128] The maximum pooling operation is performed on the contour features through convolution, and the expression is:

[0129]

[0130] Where P(x,y) is the result of the maximum pooling operation at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane, ω is the size of the window in the maximum pooling operation, i and j are the indices of the pixel positions in the window in the maximum pooling operation, and max i∈[x,x+ω-1] max j∈[y,y+ω-1] Indicates that within the maximum pooling window, for each position (x, y), find the maximum value of all positions in the window, a and b are half of the size of the convolution kernel in the horizontal and vertical directions, respectively, m and n are the indexes of the convolution kernel position in the convolution operation, I(x-i+m,y-j+n) represents the eigenvalue of the current position (x, y) combined with the offset of the convolution kernel, and K(m,n) is the weight value of the convolution kernel at position (m, n);

[0131] Back-project the contour features, the expression is:

[0132] (u,v)→K -1 [u,v,1] T →[R|t] -1 ·(K -1 [u,v,1] T )·z

[0133] Among them, (u, v) is the pixel coordinates of the contour feature for the maximum pooling operation, → represents the conversion process, K -1 is the inverse of the internal parameter matrix, [u,v,1] T is the homogeneous coordinates of a two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel;

[0134] The three-dimensional points are obtained based on the contour features after back-projection, and the obtained three-dimensional points are gathered together to form a virtual point cloud.

[0135] In this embodiment, the method for performing three-dimensional modeling according to the virtual point cloud includes:

[0136] Cluster analysis is performed on the virtual point cloud, where the objective function of the cluster analysis is expressed as:

[0137]

[0138] Among them, K is the total number of clusters, p represents the points in the virtual point cloud, and P i Represented as the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i) is the shape regularization term of clustering, α and β are weight parameters for adjusting density and shape influence respectively;

[0139] The Delaunay algorithm is used to triangulate each cluster to obtain a triangular mesh, where the triangulation constraint condition is expressed as:

[0140]

[0141] Among them, t represents a triangular patch, {p i ,p j ,p k} is expressed as the coordinates of three-dimensional space, T i is the set of all triangles in the i-th cluster, R t is the distance threshold, p represents a point in the virtual point cloud, P i Represented as the set of points in the i-th cluster, P i \{p i ,p j ,p k} means to exclude the three points that constitute the triangle patch t, ​​O t is the circumcenter of triangle patch t, ​​R t is the radius of the circumscribed circle;

[0142] The Laplace operator is introduced based on the triangulation result to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression of the solution process is:

[0143]

[0144] Among them, u represents the function to be solved, Ω is the convex hull area of ​​point p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, f is the smoothing constraint function, It is represented as the boundary of region Ω, and u0 is the boundary value;

[0145] All triangular meshes are merged to obtain a virtual simulation test scene.

[0146] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0147] The processor, the network interface and the memory may be interconnected via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an ELISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0148] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0149] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a construction device for a simulation test scenario of a smart product based on virtual reality at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the aforementioned construction methods for a simulation test scenario of a smart product based on virtual reality.

[0150] The present invention can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmab l eGate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0151] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple applications, they execute the aforementioned method for constructing any virtual reality-based smart product simulation test scenario.

[0152] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for constructing a simulation test scenario of an intelligent product based on virtual reality, characterized in that: The following steps are involved: Obtain smart product parameters and test objectives, and determine test scenario elements based on the test objectives; Perform simulation based on the parameters of the smart product to obtain a mathematical twin model of the smart product; dynamically evolve the data of the test scene elements to obtain a dynamic estimation of the test scene elements; Mapping the dynamic estimation of the test scene elements to the mathematical twin model of the smart product to obtain the mathematical twin model of the test scene; Constructing a test model based on the test purpose, simulating a test operation on the mathematical twin model of the test scene based on the test model, feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating, and obtaining a two-dimensional virtual point; Back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud; Three-dimensional modeling is performed based on the virtual point cloud to form a virtual simulation test scene for the smart product.

2. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for performing simulation based on the smart product parameters includes: Create a geometric model of the smart product using a 3D modeling tool based on the collected smart product parameters, and assign material properties to the geometric model; In the geometric model, the performance parameter equations of the smart product are distributed by the finite element method, and the differential partial equations are combined. The differential partial equations are solved by the conjugate gradient method to obtain the mathematical twin model of the smart product.

3. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for dynamically evolving data of the test scenario elements comprises: Set a time window, collect the real-time status of the test scene elements based on the time window, and obtain the time series status data of the test scene elements; Based on the convolutional neural network, the state characteristics of the test scene elements are learned from the time series state data. The convolution operation is performed on the time series state data in the time dimension through the convolution kernel. The expression is: Among them, y l [n] is the output of the lth convolutional layer at time n, M l is the number of convolution kernels in the lth layer, J l is the number of sequences that make up the lth convolutional layer, x l [nk] is the input sequence of the lth convolutional layer at time nk, g l,m,j [i] is the weight value of the jth sequence of the mth convolution kernel in the lth convolution layer at time i, f l,m,j [i+k] is the activation value of the jth sequence of the mth convolution kernel in the lth convolution layer at time i+k; The state of the convolutional neural network model is updated by the recurrent neural network based on the learned state features of the test scene elements to perform dynamic data evolution and obtain a dynamic estimate of the test scene elements.

4. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method of mapping the dynamic estimation of the test scene element to the mathematical twin model of the smart product comprises: The dynamic estimation of the test scene elements is time-aligned with the mathematical twin model of the smart product through the dynamic time warping algorithm; Each time is mapped by constructing a mapping function, where the expression of the mapping function is: F(O)=(R p +t,S(sp,θ),TS(TM,U(R p +t),V(R p +t))) Among them, F(O) is the output of the mapping function, O is the input object, and R p is the position vector, t is the time offset, i.e. R p Changes over time, S(sp,θ) is the position and direction feature function, sp is the spatial position, θ represents the direction change, TS represents the spatial model scale, TM represents the temporal model scale, U(R p +t) is expressed as the correlation function between the position vector and the time offset, V(R p +t) is represented as the transfer matrix of position vector and time offset.

5. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for constructing a test model based on the test purpose includes: Obtain directly relevant input variables based on the test purpose, and determine the value range of the input variables based on the test purpose and the design specifications of the smart product; For test purposes that can be quantified numerically, determine the quantitative output as the output of the test model. When the test purpose involves classification and / or state judgment, determine the qualitative output as the output of the test model. Construct a test function based on the test purpose, and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and a test function combination to construct a test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model combined with a test function; if the purpose of the test involves state judgment, use a neural network algorithm and a test function combination to build a test model.

6. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method of simulating a test operation on the mathematical twin model of the test scenario based on the test model, and feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scenario for updating includes: The input parameters of the test model and the current state parameters of the mathematical twin model are passed to the test model, and the mathematical twin model is simulated and tested by the test model; Extract 2D and 1D feature information from simulation test results, including the geometric boundaries and motion trajectories of smart products; Feeding the processed two-dimensional feature information and one-dimensional feature information back to the mathematical twin model of the test scene, adjusting the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and the one-dimensional feature information, and updating the state of the model; Using the updated mathematical twin model, a two-dimensional virtual point is generated in the two-dimensional space, and the expression is: Among them, f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane respectively, σ x and σ y They represent the dispersion of the two-dimensional virtual points in the x direction and the y direction respectively. ρ is the correlation coefficient between the x direction and the y direction, which means the linear correlation between the two coordinates. Its value is between -1 and 1, where 1 represents a perfect positive correlation, -1 represents a perfect negative correlation, and 0 represents no correlation.

7. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method of back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud comprises: The contour features of two-dimensional virtual points are extracted through Moore field tracking algorithm; The maximum pooling operation is performed on the contour features through convolution, and the expression is: Where P(x,y) is the result of the maximum pooling operation at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane, ω is the size of the window in the maximum pooling operation, i and j are the indices of the pixel positions in the window in the maximum pooling operation, and max i∈[x,x+ω-1] max j∈[y,y+ω-1] Indicates that within the maximum pooling window, for each position (x, y), find the maximum value of all positions in the window, a and b are half of the size of the convolution kernel in the horizontal and vertical directions, respectively, m and n are the indexes of the convolution kernel position in the convolution operation, I(x-i+m,y-j+n) represents the eigenvalue of the current position (x, y) combined with the offset of the convolution kernel, and K(m,n) is the weight value of the convolution kernel at position (m, n); Back-project the contour feature, the expression is: (u,v)→K -1 ·[u,v,1] T →[R|t] -1 ·(K -1 ·[u,v,1] T )·z Among them, (u, v) is the pixel coordinates of the contour feature for the maximum pooling operation, → represents the conversion process, K -1 is the inverse of the internal parameter matrix, [u,v,1] T is the homogeneous coordinates of a two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel; The three-dimensional points are obtained based on the contour features after back-projection, and the obtained three-dimensional points are gathered together to form a virtual point cloud.

8. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for performing three-dimensional modeling according to the virtual point cloud comprises: Cluster analysis is performed on the virtual point cloud, where the objective function of the cluster analysis is expressed as: Among them, K is the total number of clusters, p represents the points in the virtual point cloud, and P i Represented as the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i ) is the shape regularization term of clustering, α and β are weight parameters for adjusting density and shape influence respectively; The Delaunay algorithm is used to triangulate each cluster to obtain a triangular mesh, where the triangulation constraint condition is expressed as: Among them, t represents a triangular patch, {p i ,p j ,p k } is expressed as the coordinates of three-dimensional space, T i is the set of all triangles in the i-th cluster, R t is the distance threshold, p represents a point in the virtual point cloud, P i Represented as the set of points in the i-th cluster, P i \{p i ,p j ,p k } means to exclude the three points that constitute the triangle patch t, ​​O t is the circumcenter of triangle patch t, ​​R t is the radius of the circumscribed circle; The Laplace operator is introduced based on the triangulation result to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression of the solution process is: Among them, u represents the function to be solved, Ω is the convex hull area of ​​point p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, f is the smoothing constraint function, It is represented as the boundary of region Ω, and u0 is the boundary value; All triangular meshes are merged to obtain a virtual simulation test scene.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the method according to any one of claims 1 to 8.

Images