Construction Method of Intelligent Product Simulation Test Scenario Based on Virtual Reality

Abstract

The present invention discloses a method for constructing a simulation test scenario of an intelligent product based on virtual reality, including: obtaining intelligent product parameters and determining test scenario elements; obtaining the digital twin model of the intelligent product and the dynamic estimation of the scenario elements; obtaining the digital twin model of the test scenario by mapping the dynamic estimation of the scenario elements to the digital twin model of the intelligent product; performing simulation test operations on the digital twin model of the test scenario based on the test model, and feeding back the two-dimensional feature information and one-dimensional feature information output by the test model to the digital twin model for updating to obtain two-dimensional virtual points; performing back-projection based on the two-dimensional virtual points to restore the virtual point cloud; and performing three-dimensional modeling according to the virtual point cloud to form a virtual simulation test scenario of the intelligent product. This method can not only accurately simulate test scenario elements, but also has good interpretability and can be directly applied to a system for constructing a simulation test scenario of an intelligent product based on virtual reality.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality, and particularly to a method for constructing a simulation test scenario of intelligent products based on virtual reality. Background Art

[0002] By adopting a virtual reality simulation test scenario, the test cost can be reduced and the test cycle can be shortened. With virtual reality technology, various complex test scenarios can be easily constructed to comprehensively test the performance of intelligent products under different conditions.

[0003] The traditional method for constructing a test scenario of intelligent products mainly relies on the construction of an actual physical environment and physical object testing, but this method usually requires a large amount of time, manpower, and material resources.

[0004] In contrast, adopting a virtual reality simulation test scenario has obvious advantages. It can quickly construct various complex test scenarios in a virtual environment without actual physical construction, greatly reducing the test cost. At the same time, virtual reality technology can adjust the test scenario at any time according to needs, shortening the test cycle. In addition, through precise parameter settings and simulation algorithms, the performance of intelligent products under different conditions can be tested more comprehensively and accurately, improving the reliability and effectiveness of the test. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for constructing a simulation test scenario of intelligent products based on virtual reality.

[0006] To achieve the above object, the present invention is implemented according to the following technical solution:

[0007] The first aspect of the present invention provides a method for constructing a simulation test scenario of intelligent products based on virtual reality, including the following steps:

[0008] S100 Obtain intelligent product parameters and test purposes, and determine test scenario elements based on the test purposes;

[0009] S200 Perform simulation based on the intelligent product parameters to obtain a digital twin model of the intelligent product; perform data dynamic evolution on the test scenario elements to obtain a dynamic estimate of the test scenario elements;

[0010] S300 Map the dynamic estimate of the test scenario elements to the digital twin model of the intelligent product to obtain a digital twin model of the test scenario;

[0011] The S400 constructs a test model based on the test purpose, performs simulation test operations on the mathematical twin model of the test scenario based on the test model, feeds back the two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twin model of the test scenario for updating, and obtains two-dimensional virtual points;

[0012] S500 back-projects the two-dimensional virtual points to a three-dimensional coordinate system to restore the virtual point cloud;

[0013] S600 performs three-dimensional modeling based on the virtual point cloud to form a virtual simulation test scenario of the intelligent product.

[0014] As a further method, the method for performing simulation and emulation based on the parameters of the intelligent product includes:

[0015] Create a geometric model of the intelligent product through a three-dimensional modeling tool according to the collected parameters of the intelligent product, and assign material properties to the geometric model;

[0016] In the geometric model, assign the performance parameter equations of the intelligent product through the finite element method, combine to obtain the partial differential equation, and solve the partial differential equation through the conjugate gradient method to obtain the mathematical twin model of the intelligent product.

[0017] As a further method, the method for performing data dynamic evolution on the test scenario elements includes:

[0018] Set a time window, collect the real-time state of the test scenario elements based on the time window, and obtain the time-series state data of the test scenario elements;

[0019] Learn the state features of the test scenario elements from the time-series state data based on the convolutional neural network, perform convolutional operations on the time-series state data in the time dimension through the convolutional kernel, and the expression is:

[0020]

[0021] where, y l [n] is the output of the l-th convolutional layer at time n, M l is the number of convolutional kernels in the l-th layer, J l is the number of sequences constituting the l-th convolutional layer, x l [n-k] is the input sequence of the l-th convolutional layer at time n-k, gl,m,j[i] is the weight value of the j-th sequence of the m-th convolutional kernel in the l-th convolutional layer at time i, and fl,m,j[i+k] is the activation value of the j-th sequence of the m-th convolutional kernel in the l-th convolutional layer at time i+k;

[0022] Data dynamic evolution is performed by updating the state of the convolutional neural network model based on the state features of the learned test scenario elements through a recurrent neural network, and a dynamic estimate of the test scenario elements is obtained.

[0023] As a further method, the method of mapping the dynamic estimate of the test scenario elements to the digital twin model of the intelligent product includes:

[0024] Align the time of the dynamic estimate of the test scenario elements with the digital twin model of the intelligent product through the dynamic time warping algorithm;

[0025] Map each time through constructing a mapping function, where the expression of the mapping function is:

[0026] F(O)(Rp+t,S(sp,0),TS(TM,U(Rp+t),V(R,+t)))

[0027] Where, F(O) is the output of the mapping function, O is the input object, Rp is the position vector, t is the time offset, i.e., the change of Rp over time, S(sp,θ) is the position and orientation feature function, sp is the spatial position, θ represents the orientation change, TS represents the spatial model scale, TM represents the time model scale, U(Rp+t) represents the correlation function between the position vector and the time offset, and V(Rp+t) represents the transfer matrix between the position vector and the time offset.

[0028] As a further method, the method of constructing a test model based on the test purpose includes:

[0029] Obtain directly relevant input variables based on the test purpose, and determine the value range of the input variables based on the test purpose and the design specifications of the intelligent product;

[0030] For test purposes that can be numerically quantified, determine the quantitative output as the output of the test model. When the test purpose involves classification and / or state judgment, determine the qualitative output as the output of the test model;

[0031] Build a test function based on the test purpose, and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and a test function combination to build the test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model in combination with the test function, and if the test purpose involves state judgment, use a neural network algorithm to build a test model in combination with the test function.

[0032] As a further method, the method of simulating a test operation on the mathematical twin model of the test scenario based on the test model and feeding back the two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twin model of the test scenario for updating includes:

[0033] Transfer the input parameters of the test model and the current state parameters of the mathematical twin model to the test model, and perform a simulation test operation on the mathematical twin model through the test model;

[0034] Extract two-dimensional feature information and one-dimensional feature information from the simulation test results, including the geometric boundaries and motion trajectories of intelligent products;

[0035] Feed back the processed two-dimensional feature information and one-dimensional feature information to the mathematical twin model of the test scenario, and adjust the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and one-dimensional feature information to update the state of the model;

[0036] Generate two-dimensional virtual points in the two-dimensional space by using the updated mathematical twin model, and the expression is:

[0037]

[0038] where f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y respectively represent the abscissa and ordinate on the two-dimensional plane, and σ x and σ y respectively represent the dispersion degree of the two-dimensional virtual point in the x direction and the y direction, ρ is the correlation coefficient between the x direction and the y direction, that is, it represents the linear correlation between the two coordinates, and its value is between -1 and 1, where 1 represents a perfect positive correlation, -1 represents a perfect negative correlation, and 0 represents no correlation.

[0039] As a further method, the method of back-projecting the two-dimensional virtual points to the three-dimensional coordinate system to restore the virtual point cloud includes:

[0040] Extract contour features from the two-dimensional virtual points through the Moore neighborhood tracking algorithm;

[0041] Perform a max-pooling operation calculation on the contour features through convolution, and the expression is:

[0042]

[0043] where P(x,y) is the result after the max-pooling operation at the coordinate (x,y), x and y respectively represent the abscissa and ordinate on the two-dimensional plane, ω is the size of the window in the max-pooling operation, i and j are the indexes of the pixel positions in the window in the max-pooling operation, and max i∈[x,x+ω-1]max j∈[y,y+ω-1] It means that within the maximum pooling window, for each position (x, y), the maximum value among all positions within the window is found. a and b are respectively half of the sizes of the convolutional kernel in the horizontal and vertical directions, m and n are the indices of the convolutional kernel positions in the convolution operation, I(x - i + m, y - j + n) represents the eigenvalue combined with the offset of the current position (x, y) and the convolutional kernel, and K(m, n) is the weight value of the convolutional kernel at the position (m, n);

[0044] Back-project the contour features, and the expression is:

[0045] (u, v) → K -1 · [u, v, 1] T → [R|t] -1 · (K -1 · [u, v, 1] T ) · z

[0046] Among them, (u, v) are the pixel coordinates calculated by the maximum pooling operation on the contour features, → represents the conversion process, K -1 is the inverse of the intrinsic matrix, [u, v, 1] T is the homogeneous coordinate of the two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel point;

[0047] Obtain three-dimensional points based on the back-projected contour features, and gather the obtained three-dimensional points to form a virtual point cloud.

[0048] As a further method, the method for three-dimensional modeling according to the virtual point cloud includes:

[0049] Perform clustering analysis on the virtual point cloud, and the expression of the objective function of the clustering analysis is:

[0050]

[0051] Among them, K is the total number of clusters, p represents the points in the virtual point cloud, P i represents the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i ) is the shape regularization term of the cluster, and α and β are respectively the weight parameters for adjusting the density and shape effects;

[0052] Perform triangulation processing within each cluster through the Delaunay algorithm to obtain a triangular mesh, and the expression of the constraint conditions for the triangulation processing is:

[0053]

[0054] Among them, t represents a triangular patch, and {p i , p j , p k} represents the coordinates in three-dimensional space. T i is the set of all triangular patches in the i-th cluster, R t is the distance threshold, p represents a point in the virtual point cloud, and P i represents the set of points in the i-th cluster. P i \{p i , p j , p k} represents excluding the three points that make up the triangular patch t. O t is the circumcenter of the triangular patch t, and R t is the radius of the circumcircle;

[0055] Using the result of triangulation, the Laplace operator is introduced to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression in the solution process is:

[0056]

[0057] Among them, u represents the function to be solved, Ω is the convex hull region of the point p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, and f is the smoothing constraint function. represents the boundary of the region Ω, and u0 is the boundary value;

[0058] All the triangular meshes are merged to obtain a virtual simulation test scenario.

[0059] In a second aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the method steps described in the first aspect.

[0060] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute the method steps described in the first aspect.

[0061] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0062] (1) Starting from obtaining the intelligent product parameters and test purposes, the present invention can accurately determine the test scenario elements based on the test purposes, closely linking the product with the scenario. This overall consideration can more comprehensively evaluate the scenario situation of the product in actual use.

[0063] (2) By dynamically evolving the data of the test scenario elements and mapping them into the mathematical twin model of the intelligent product, the present invention can more realistically reflect the performance of the product in a complex and changeable environment.

[0064] (3) The present invention restores the virtual point cloud by back-projecting the two-dimensional virtual points, and then performs three-dimensional modeling based on the virtual point cloud to form a virtual simulation test scenario. The two-dimensional virtual points can be used as an intermediate representation of the three-dimensional model, which helps to extract useful two-dimensional feature information from the complex three-dimensional structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flowchart of the steps of the method for constructing a simulation test scenario of an intelligent product based on virtual reality according to the present invention.

[0066] Figure 2 It is a schematic structural diagram of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] Referring to Figure 1 as shown, the present invention provides a method for constructing a simulation test scenario of an intelligent product based on virtual reality, including:

[0069] S100 Obtain the intelligent product parameters and the test purpose, and determine the test scenario elements based on the test purpose;

[0070] It should be noted that the intelligent product parameter data is a quantitative description of the product characteristics, and these data can be obtained from product manuals, technical specification documents, data interfaces provided by manufacturers, or professional test equipment, etc. The test purpose is the starting point and the end point of the intelligent product test, which clarifies the test objectives and expected results. The test purpose can be determined according to different requirements and application scenarios. For example: (1) Product performance test, to test the performance of the intelligent product under various conditions, such as processing speed, response time, battery life, etc. For example, to test the processor performance of an intelligent tablet computer, various applications and games can be run to measure its running speed and smoothness; (2) Reliability test: To evaluate the reliability and stability of the intelligent product during long-term use. Different usage environments and working conditions can be simulated for long-term testing to observe whether the product fails or its performance degrades.

[0071] It should be understood that test scenario elements refer to various factors that need to be considered during the testing process. Together with the product to be tested, they constitute a complete test scenario. Depending on the different testing purposes, the test scenario elements will also vary;

[0072] In the actual evaluation, obtain the parameter data of the smart bracelet, including:

[0073] Physical parameters: The main body of the bracelet has dimensions of 40mm in length, 30mm in width, and 10mm in thickness. The screen diameter is 20mm. The strap has a length of 200mm, a width of 15mm, and a thickness of 2mm, and a weight of 30g;

[0074] Functional parameters: It is equipped with an acceleration sensor, a heart rate sensor, and a Bluetooth communication module. The acceleration sensor has a measurement range of ±8g (where g is the acceleration due to gravity, g = 9.8m / s 2 ), the heart rate sensor has a measurement range of 30 - 220 beats per minute, and the Bluetooth communication distance is within 10m. The battery capacity is 100mAh, and the battery life is 7 days in normal usage mode;

[0075] The purpose of this test is to test the accuracy and stability of the health monitoring functions (heart rate, number of steps) of the smart bracelet in different exercise states and environments, as well as the stability of Bluetooth connection;

[0076] In the actual evaluation, determine the test scenario elements, including:

[0077] Related to exercise states: Static: The body is stationary and the arms hang naturally; Slow walking: The speed is about 3 - 4km / h and the arms swing normally; Fast walking: The speed is about 5 - 6km / h and the arms swing with a larger amplitude; Running: The speed is about 8 - 10km / h and the arms swing rhythmically and quickly;

[0078] Related to environments: Indoor environment: The temperature is 22 - 25°C, the relative humidity is 40 - 60%, and there is no electromagnetic interference source (such as far from strong interference devices like microwave ovens and wireless routers); Outdoor environment: The temperature is 15 - 30°C, the relative humidity is 30 - 70%, and there is certain electromagnetic interference (such as near base stations, public Wi-Fi hotspots, etc.);

[0079] S200 performs simulation based on the smart product parameters to obtain a digital twin model of the smart product; performs data dynamic evolution on the test scenario elements to obtain a dynamic estimation of the test scenario elements;

[0080] It should be noted that the dynamic estimation of scenario elements refers to the quantitative description of various elements required to construct a test scenario (such as environmental elements, external interferences, user behaviors, etc.) over time. For example, when testing an intelligent drone, the scenario elements may include wind speed, wind direction, air density (environmental factors), interference from other surrounding flying objects (external interferences), and the flight instructions of the operator (user behaviors), etc. These elements are not static but constantly changing, and the dynamic estimation of them is to determine the values or variation rules of these elements at different moments or in different states;

[0081] It should be noted that in this way, the dynamic changes of the test scenario in the time dimension can be simulated, so as to obtain the dynamic estimation of scenario elements. The purpose of doing this is to more realistically simulate the dynamic changes of various scenario elements in the actual use environment. Intelligent products often face various dynamically changing environmental conditions in actual use. Through this dynamic estimation, the performance, stability, and adaptability of the product in different dynamic scenarios can be better evaluated. For example, it can be observed whether the performance of the intelligent product is affected during the process of gradually increasing temperature, or whether it can maintain a stable connection and normal functions under the condition of network delay fluctuations;

[0082] In actual evaluation, the SolidWorks software is used to create a three-dimensional geometric model, and further material properties are assigned to the three-dimensional geometric model through the marginal material option. For polycarbonate, its density is set to 1.2 g / cm 3 , the elastic modulus is 2.4 GPa, and the Poisson's ratio is 0.38; for silicone, the density is set to 1.1 g / cm 3 , the elastic modulus is 0.01 GPa, and the Poisson's ratio is 0.49;

[0083] In actual evaluation, according to the working principle of the heart rate sensor, heart rate monitoring is based on photoplethysmography (PPG). There is a relationship between the PPG signal y(t) and the heart rate HR: HR = f(y(t)), where f is a function based on signal processing algorithms. The heart rate is calculated by performing operations such as filtering and peak detection on the PPG signal, where T is the time interval (unit: second) between two adjacent PPG signal peaks; for the monitoring of the number of exercise steps, according to the data of the acceleration sensor, the acceleration data on three axes (x, y, z) are obtained as a x (t), a y (t), a z (t), and the resultant acceleration When a(t) exceeds the threshold of 0.5g and meets certain time intervals and waveform characteristics, it is determined as one step; for the Bluetooth connection stability, according to the Bluetooth signal transmission model, the relationship between the signal strength P(d) and the transmission distance d is: where P0 is the signal strength at the reference distance d0, and n is the path loss exponent (n = 2 indoors and n = 3 outdoors); the equation is discretized by the finite element method and then solved using the conjugate gradient method to obtain the variation of the temperature distribution with time and space, thereby constructing the mathematical twin model of the smart bracelet;

[0084] In the actual evaluation, the data in the motion state is dynamically evolved as follows: Slow walking scenario: Arm swing angle: The swing angle θ(t) of the arm in the horizontal direction changes sinusoidally with time t, (The angle unit is degrees, the unit of t is seconds, and one period is 4 seconds); Acceleration data: According to the kinematic principle, when walking slowly, the acceleration a arm (t) will have periodic changes, a arm (t) = 0.2gsin(πt); Fast walking scenario: Arm swing angle The swing amplitude is larger than that of slow walking; Acceleration data: a arm (t) = 0.3gsin(πt); Running scenario: Arm swing angle Acceleration data: a arm (t) = 0.5gsin(πt);

[0085] In the actual evaluation, the indoor environmental temperature change: The indoor temperature T indoor (t) has a small amplitude fluctuation within a day, (The unit of t is hours); Outdoor environmental temperature change: The fluctuation amplitude is larger than that indoors; Electromagnetic interference intensity change (outdoor scenario): The electromagnetic interference intensity I(t) is related to the signal transmission period of the nearby base station, where I0 is the average interference intensity;

[0086] S300 maps the dynamic estimation of the test scenario elements to the mathematical twin model of the smart product to obtain the mathematical twin model of the test scenario;

[0087] It should be noted that mapping the dynamic estimation of scene elements to the digital twin model of an intelligent product means establishing the mutual relationship between scene elements and the intelligent product. In the example of an intelligent drone, mapping the dynamic estimation of environmental factors such as wind speed and wind direction to the digital twin model of the drone means considering the influence of these environmental factors in the dynamic equation of the drone. For example, wind speed affects the flight speed and attitude of the drone, and wind direction changes the flight direction of the drone. These effects can be reflected by adding terms related to wind speed and wind direction to the force balance equation of the drone;

[0088] It should be noted that through this mapping process, the digital twin model of the test scene obtained is not just the model of the intelligent product itself, but a comprehensive model that includes the influence of the test scene on the product. This model can be used to simulate the behavior and performance of intelligent products in various scenarios. Continuing with the example of an intelligent drone, this comprehensive model can be used to simulate performance indicators such as the flight trajectory, energy consumption, and stability of the drone under different wind speeds, wind directions, and user operations, thus providing a strong basis for the testing, optimized design, and fault prediction of the drone;

[0089] In actual evaluation, in the digital twin model of an intelligent bracelet, for the part of monitoring the number of steps, the arm swing angle θ(t) and acceleration a arm (t) are mapped to the input of the acceleration sensor. According to the installation position and direction of the bracelet, the actual motion acceleration of the arm is converted into the acceleration component in the direction of the acceleration sensor coordinate axis through coordinate transformation, thus affecting the calculation of the number of steps; for heart rate monitoring, changes in the motion state will affect the PPG signal. When the arm swings, due to changes in blood flow and the light propagation path, the PPG signal will be interfered. When the arm swing angle is θ(t), the amplitude of the PPG signal will change according to the relationship y(t) = y0(1 - 0.05sin(θ(t))), where y0 is the amplitude of the PPG signal at rest;

[0090] In actual evaluation, in the part of Bluetooth connection stability, the electromagnetic interference intensity I(t) in the outdoor environment is mapped to the signal transmission model of the Bluetooth communication module. According to the influence model of electromagnetic interference on Bluetooth signals, the interference intensity I(t) will cause an increase in the noise of the Bluetooth signal, thus changing the calculation of the signal strength P(d), and the signal strength becomes where k is the interference coefficient; for temperature changes, the indoor and outdoor environmental temperatures T indoor (t) and T outdoor (t) are mapped to the heat conduction model of the intelligent bracelet;

[0091] The S400 constructs a test model based on the test purpose, performs simulation test operations on the digital twin model of the test scenario based on the test model, and feeds back the two-dimensional feature information and one-dimensional feature information output by the test model to the digital twin model of the test scenario for updating to obtain two-dimensional virtual points;

[0092] It should be noted that different test purposes determine the elements and features that need to be emphasized in the virtual scenario. For example, if the test purpose is to evaluate the collaborative efficiency of robots in an intelligent factory, then when constructing the virtual scenario, it is necessary to accurately depict the layout of the factory, the movement trajectories of the robots, and their interaction methods. By clarifying the test purpose, the details of the virtual scenario can be designed in a targeted manner to make it more in line with the actual test requirements;

[0093] It should be noted that performing simulation test operations on the digital twin model of the test scenario can provide a more realistic dynamic effect for the virtual scenario. The digital twin model combines the physical laws in the actual scenario and the characteristics of intelligent products. Through simulation test operations, the state in the virtual scenario can be updated in real time, making the virtual scenario closer to the actual situation. For example, in the virtual scenario of an intelligent building, by performing simulation tests on the digital twin model, parameters such as the energy consumption, temperature, and lighting of the building can be dynamically adjusted according to factors such as different weather conditions and personnel activities, making the virtual scenario more realistic and reliable;

[0094] In actual evaluation, according to the set test scenario elements (different motion states and environments), corresponding parameters are input into the test model. For example, in the slow walking scenario, the previously calculated arm swing angle and acceleration are converted into the actual input data of the acceleration sensor, and then the operation of the test model is performed to obtain monitoring results such as the number of steps and heart rate;

[0095] The S500 performs back-projection on the two-dimensional virtual points to restore the virtual point cloud in the three-dimensional coordinate system;

[0096] It should be understood that restoring the virtual point cloud is to reconstruct a complete three-dimensional representation of the scene or object. The point cloud is a set composed of a large number of three-dimensional points, which can more comprehensively and accurately describe the test scenario where the intelligent product is located or the three-dimensional shape of the intelligent product itself. By restoring the virtual point cloud, three-dimensional modeling can be further carried out, the spatial relationship between the intelligent product and the scene can be analyzed, and a basis for more accurate performance testing and evaluation can be provided in the future;

[0097] In actual evaluation, the screen plane of the smart bracelet is a two-dimensional plane. A two-dimensional coordinate system (x, y) is established with the center of the screen as the origin. The position of the bracelet in the three-dimensional space can be determined by its wearing position on the human arm, and the initial position of the bracelet in the three-dimensional space can be determined according to the position of the arm where the bracelet is worn. Subsequently, through back-projection calculation, the two-dimensional virtual points can be restored to the three-dimensional coordinate system to obtain multiple three-dimensional points, and these points form a virtual point cloud.

[0098] S600 performs three-dimensional modeling based on the virtual point cloud to form a virtual simulation test scenario for the smart product.

[0099] In actual evaluation, performance testing is performed in the simulation test scenario.

[0100] Specifically, set the initial state of the smart bracelet in the scenario, such as battery power, Bluetooth connection status, etc. In this way, a virtual simulation test scenario for the smart bracelet is formed, and the performance of the smart bracelet under different test conditions can be further observed and analyzed in this scenario.

[0101] In this embodiment, the method for performing simulation and emulation based on the smart product parameters includes:

[0102] Create a geometric model of the smart product through a three-dimensional modeling tool according to the collected smart product parameters, and assign material properties to the geometric model.

[0103] In the geometric model, assign the performance parameter equations of the smart product through the finite element method, combine them to obtain a micro-partial differential equation, and solve the micro-partial differential equation through the conjugate gradient method to obtain the mathematical twin model of the smart product.

[0104] In this embodiment, the method for dynamically evolving the data of the test scenario elements includes:

[0105] Set a time window, collect the real-time state of the test scenario elements based on the time window, and obtain the time-series state data of the test scenario elements.

[0106] Learn the state characteristics of the test scenario elements from the time-series state data based on a convolutional neural network, and perform a convolution operation on the time-series state data in the time dimension through a convolution kernel. The expression is:

[0107]

[0108] where, y l [n] is the output of the l-th convolutional layer at time n, M l is the number of convolutional kernels of the l-th layer, J l is the number of sequences constituting the l-th convolutional layer, x l[n - k] is the input sequence of the l-th convolutional layer at time n - k, gl,m,j[i] is the weight value of the j-th sequence of the m-th convolutional kernel in the l-th convolutional layer at time i, and fl,m,j[i + k] is the activation value of the j-th sequence of the m-th convolutional kernel in the l-th convolutional layer at time i + k;

[0109] The state of the convolutional neural network model is updated based on the learned state features of the test scenario elements through a recurrent neural network to perform data dynamic evolution, and a dynamic estimate of the test scenario elements is obtained.

[0110] In this embodiment, the method for mapping the dynamic estimate of the test scenario elements to the digital twin model of the intelligent product includes:

[0111] The dynamic estimate of the test scenario elements and the digital twin model of the intelligent product are time-aligned through the dynamic time warping algorithm;

[0112] A mapping function is constructed to perform mapping for each time, and the expression of the mapping function is:

[0113] F(0) = (Rp + t, S(sp, 0), Ts(TM, U(R, +t), V(R, +t)))

[0114] where F(O) is the output of the mapping function, O is the input object, Rp is the position vector, t is the time offset, i.e., the change of Rp over time, S(sp, θ) is the position and direction feature function, sp is the spatial position, θ represents the direction change, TS represents the spatial model scale, TM represents the time model scale, U(Rp + t) represents the correlation function of the position vector and the time offset, and V(R p + t) represents the transfer matrix of the position vector and the time offset.

[0115] In this embodiment, the method for constructing a test model based on the test purpose includes:

[0116] Input variables directly related to the test purpose are obtained based on the test purpose, and the value range of the input variables is determined based on the test purpose and the design specifications of the intelligent product;

[0117] For test purposes that can use numerical quantification, a quantitative output is determined as the output of the test model. When the test purpose involves classification or / and state judgment, a qualitative output is determined as the output of the test model;

[0118] Build a test function for testing purposes and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and combine it with the test function to build the test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model in combination with the test function, and if the test purpose involves state judgment, use a neural network algorithm and combine it with the test function to build the test model.

[0119] In this embodiment, the method of simulating a test operation on the mathematical twin model of the test scenario based on the test model and feeding back the two-dimensional feature information and one-dimensional feature information output by the test model to the mathematical twin model of the test scenario for updating includes:

[0120] Transfer the input parameters of the test model and the current state parameters of the mathematical twin model to the test model, and perform a simulation test operation on the mathematical twin model through the test model;

[0121] Extract two-dimensional feature information and one-dimensional feature information from the simulation test results, including the geometric boundary and motion trajectory of the intelligent product;

[0122] Feed back the processed two-dimensional feature information and one-dimensional feature information to the mathematical twin model of the test scenario, and adjust the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and one-dimensional feature information to update the state of the model;

[0123] Generate two-dimensional virtual points in the two-dimensional space by using the updated mathematical twin model. The expression is:

[0124]

[0125] where f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y respectively represent the abscissa and ordinate on the two-dimensional plane, and σ x and σ y respectively represent the dispersion degree of the two-dimensional virtual point in the x direction and the y direction, ρ is the correlation coefficient between the x direction and the y direction, which represents the linear correlation between the two coordinates, and its value is between -1 and 1, where 1 represents a perfect positive correlation, -1 represents a perfect negative correlation, and 0 represents no correlation.

[0126] In this embodiment, the method of back-projecting the two-dimensional virtual points to the three-dimensional coordinate system to restore the virtual point cloud includes:

[0127] Extract contour features from the two-dimensional virtual points through the Moore domain tracking algorithm;

[0128] Perform a maximum pooling operation calculation on the contour features through convolution. The expression is:

[0129]

[0130] Among them, P(x, y) is the result of the max-pooling operation at the coordinate (x, y), where x and y represent the abscissa and ordinate on the two-dimensional plane respectively, ω is the size of the window in the max-pooling operation, i and j are the indices of the pixel positions within the window in the max-pooling operation, and max i∈[x,x+ω-1] max j∈[y,y+ω-1] means that within the max-pooling window, for each position (x, y), the maximum value among all positions within the window is found. a and b are respectively half of the sizes of the convolutional kernel in the horizontal and vertical directions, m and n are the indices of the convolutional kernel positions in the convolutional operation, I(x - i + m, y - j + n) represents the eigenvalue combined with the offset of the current position (x, y) and the convolutional kernel, and K(m, n) is the weight value of the convolutional kernel at the position (m, n);

[0131] Back-project the contour features, and the expression is:

[0132] (u, v) → K -1 · [u, v, 1] T → [R|t] -1 · (K -1 · [u, v, 1] T ) · z

[0133] Among them, (u, v) are the pixel coordinates calculated by the max-pooling operation on the contour features, → represents the conversion process, K -1 is the inverse of the intrinsic matrix, [u, v, 1] T is the homogeneous coordinate of the two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel point;

[0134] Obtain three-dimensional points based on the back-projected contour features, and gather the obtained three-dimensional points to form a virtual point cloud.

[0135] In this embodiment, the method for three-dimensional modeling according to the virtual point cloud includes:

[0136] Perform clustering analysis on the virtual point cloud, and the expression of the objective function of the clustering analysis is:

[0137]

[0138] Among them, K is the total number of clusters, p represents the points in the virtual point cloud, P i represents the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i) is the shape regularization term for clustering, and α and β are the weight parameters for adjusting the density and shape influence respectively;

[0139] Inside each cluster, triangulation is performed through the Delaunay algorithm to obtain a triangular mesh. The expression for the constraints of the triangulation is:

[0140]

[0141] where t represents a triangular patch, {p i , p j , p k} represents the coordinates in three-dimensional space, T i is the set of all triangular patches in the i-th cluster, R t is the distance threshold, p represents a point in the virtual point cloud, P i represents the set of points in the i-th cluster, P i \{p i , p j , p k} represents excluding the three points that form the triangular patch t, O t is the circumcenter of the triangular patch t, R t is the radius of the circumcircle;

[0142] Using the result of triangulation, the Laplace operator is introduced to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression for the solution process is:

[0143]

[0144] where u represents the function to be solved, Ω is the convex hull region of the points p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, f is the smoothing constraint function, represents the boundary of the region Ω, and u0 is the boundary value;

[0145] All the triangular meshes are merged to obtain a virtual simulation test scenario.

[0146] Figure 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 2 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-vo l at i l e memory), such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0147] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0148] Memory, which is used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0149] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a construction device for an intelligent product simulation test scenario based on virtual reality at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the foregoing construction methods for an intelligent product simulation test scenario based on virtual reality.

[0150] The present invention can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0151] The embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including a plurality of application programs, execute any of the foregoing methods for constructing a simulation test scenario of a virtual reality-based intelligent product.

[0152] The above content is only an example and explanation of the structure of the present invention. Those skilled in the art of the present technology can make various modifications, supplements, or use similar methods to replace the specific embodiments described, as long as they do not deviate from the structure of the invention or exceed the scope defined by the claims of the present invention, they should fall within the protection scope of the present invention.

Claims

1. A method for constructing a simulation test scenario of an intelligent product based on virtual reality, characterized in that: The following steps are involved: Obtain smart product parameters and test objectives, and determine test scenario elements based on the test objectives; Perform simulation based on the parameters of the smart product to obtain a mathematical twin model of the smart product; dynamically evolve the data of the test scene elements to obtain a dynamic estimation of the test scene elements; Mapping the dynamic estimation of the test scene elements to the mathematical twin model of the smart product to obtain the mathematical twin model of the test scene; Constructing a test model based on the test purpose, simulating a test operation on the mathematical twin model of the test scene based on the test model, feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scene for updating, and obtaining a two-dimensional virtual point; Back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud; Perform three-dimensional modeling based on the virtual point cloud to form a virtual simulation test scene for the smart product; The method of mapping the dynamic estimation of the test scene element to the mathematical twin model of the smart product comprises: The dynamic estimation of the test scene elements is time-aligned with the mathematical twin model of the smart product through the dynamic time warping algorithm; Each time is mapped by constructing a mapping function, where the expression of the mapping function is: F(O)=(R p +t,S(sp,θ),TS(TM,U(R p +t),V(R p +t))) Among them, F(O) is the output of the mapping function, O is the input object, and R p is the position vector, t is the time offset R p Changes over time, S(sp,θ) is the position and direction feature function, sp is the spatial position, θ represents the direction change, TS represents the spatial model scale, TM represents the temporal model scale, U(R p +t) is expressed as the correlation function between the position vector and the time offset, V(R p +t) is represented as the transfer matrix of position vector and time offset; The method of back-projecting the two-dimensional virtual points into a three-dimensional coordinate system to restore a virtual point cloud comprises: The contour features of two-dimensional virtual points are extracted through Moore field tracking algorithm; The maximum pooling operation is performed on the contour features through convolution, and the expression is: Where P(x,y) is the result of the maximum pooling operation at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane, ω is the size of the window in the maximum pooling operation, i and j are the indices of the pixel positions in the window in the maximum pooling operation, and max i∈[x,x+ω-1] max j∈[y,y+ω-1] Indicates that within the maximum pooling window, for each position (x, y), find the maximum value of all positions in the window, a and b are half of the size of the convolution kernel in the horizontal and vertical directions, respectively, m and n are the indexes of the convolution kernel position in the convolution operation, I(x-i+m,y-j+n) represents the eigenvalue of the current position (x, y) combined with the offset of the convolution kernel, and K(m,n) is the weight value of the convolution kernel at position (m, n); Back-project the contour features, the expression is: (u,v)→K -1 ·[u,v,1] T →[R|t] -1 ·(K -1 ·[u,v,1] T )·z Among them, (u, v) is the pixel coordinates of the contour feature for the maximum pooling operation, → represents the conversion process, K -1 is the inverse of the internal parameter matrix, [u,v,1] T is the homogeneous coordinates of a two-dimensional virtual point, [R|t] -1 is the inverse of the extrinsic matrix, and z is the depth value of the pixel; The three-dimensional points are obtained based on the contour features after back-projection, and the obtained three-dimensional points are gathered together to form a virtual point cloud.

2. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for performing simulation based on the smart product parameters includes: Create a geometric model of the smart product using a 3D modeling tool based on the collected smart product parameters, and assign material properties to the geometric model; In the geometric model, the performance parameter equations of the smart product are distributed by the finite element method, and the differential partial equations are combined. The differential partial equations are solved by the conjugate gradient method to obtain the mathematical twin model of the smart product.

3. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for dynamically evolving data of the test scenario elements comprises: Set a time window, collect the real-time status of the test scene elements based on the time window, and obtain the time series status data of the test scene elements; Based on the convolutional neural network, the state characteristics of the test scene elements are learned from the time series state data. The convolution operation is performed on the time series state data in the time dimension through the convolution kernel. The expression is: Among them, y l [n] is the output of the lth convolutional layer at time n, M l is the number of convolution kernels in the lth layer, J l is the number of sequences that make up the lth convolutional layer, x l [nk] is the input sequence of the lth convolutional layer at time nk, g l,m,j [i] is the weight value of the jth sequence of the mth convolution kernel in the lth convolution layer at time i, f l,m,j [i+k] is the activation value of the jth sequence of the mth convolution kernel in the lth convolution layer at time i+k; The state of the convolutional neural network model is updated by the recurrent neural network based on the learned state features of the test scene elements to perform dynamic data evolution and obtain a dynamic estimate of the test scene elements.

4. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for constructing a test model based on the test purpose includes: Obtain directly relevant input variables based on the test purpose, and determine the value range of the input variables based on the test purpose and the design specifications of the smart product; For test purposes that can be quantified numerically, determine the quantitative output as the output of the test model. When the test purpose involves classification and / or state judgment, determine the qualitative output as the output of the test model. Construct a test function based on the test purpose, and build a test model through machine learning methods. Specifically: when the output of the test model is quantitative, select a regression algorithm and a test function combination to construct a test model; when the output of the test model is qualitative, if the test purpose involves classification, use a decision tree algorithm to build a test model combined with a test function; if the purpose of the test involves state judgment, use a neural network algorithm and a test function combination to build a test model.

5. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method of simulating a test operation on the mathematical twin model of the test scenario based on the test model, and feeding back the two-dimensional feature information and the one-dimensional feature information output by the test model to the mathematical twin model of the test scenario for updating includes: The input parameters of the test model and the current state parameters of the mathematical twin model are passed to the test model, and the mathematical twin model is simulated and tested by the test model; Extract 2D and 1D feature information from simulation test results, including the geometric boundaries and motion trajectories of smart products; Feeding the processed two-dimensional feature information and one-dimensional feature information back to the mathematical twin model of the test scene, adjusting the geometric shape and motion equation parameters of the model according to the geometric boundary and motion trajectory information in the two-dimensional feature information and the one-dimensional feature information, and updating the state of the model; Using the updated mathematical twin model, a two-dimensional virtual point is generated in the two-dimensional space, and the expression is: Among them, f(x,y) represents the probability density of generating a two-dimensional virtual point at the coordinate (x,y), x and y represent the horizontal and vertical coordinates on the two-dimensional plane respectively, σ x and σ y They are respectively represented as the dispersion degree of the two-dimensional virtual point in the x direction and the y direction, ρ is the correlation coefficient between the x direction and the y direction, indicating the linear correlation between the two coordinates, and its value is between -1 and 1, where 1 indicates a complete positive correlation, -1 indicates a complete negative correlation, and 0 indicates no correlation.

6. The method for constructing a virtual reality-based intelligent product simulation test scenario according to claim 1, characterized in that: The method for performing three-dimensional modeling according to the virtual point cloud comprises: Cluster analysis is performed on the virtual point cloud, where the objective function of the cluster analysis is expressed as: Where K is the total number of clusters, p represents the points in the virtual point cloud, and P i Represented as the set of points in the i-th cluster, μ i is the center of the cluster, d(p) is the local density of the virtual point cloud p, s(P i ) is the shape regularization term of clustering, α and β are weight parameters for adjusting density and shape influence respectively; The Delaunay algorithm is used to triangulate each cluster to obtain a triangular mesh, where the triangulation constraint condition is expressed as: Among them, t represents a triangular face, {p i ,p j ,p k } is expressed as the coordinates of three-dimensional space, T i is the set of all triangles in the i-th cluster, R t is the distance threshold, P i \{p i ,p j ,p k } means to exclude the three points that constitute the triangle patch t, ​​O t is the circumcenter of triangle patch t, ​​R t is the radius of the circumscribed circle; The Laplace operator is introduced based on the triangulation result to solve the Poisson equation to reconstruct the surface of the triangular mesh. The expression of the solution process is: Among them, u represents the function to be solved, Ω is the convex hull area of ​​point p in the virtual point cloud, is the gradient of u, λ is the smoothing parameter, f is the smoothing constraint function, It is represented as the boundary of region Ω, and u0 is the boundary value; All triangular meshes are merged to obtain a virtual simulation test scene.

7. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the method according to any one of claims 1 to 6.

Images