A virtual simulation method based on self-learning

By adopting self-learning methods in virtual simulation technology, analyzing the brightness and shape of the optical environment, and automatically detecting mark point occlusion and light source interference, the problem of inaccurate data acquisition in virtual simulation is solved, and the virtual simulation effect and data acquisition accuracy are improved sex.

CN118819308BInactive Publication Date: 2024-12-06JINGGANGSHAN UNIVERSITY
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411314049.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-20
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing virtual simulation technology lacks accurate identification and analysis of abnormal data of marking point data, resulting in inaccurate data acquisition and affecting the virtual simulation effect.

Method used

By analyzing the brightness, shape and relative position of the optical environment, we automatically detect marking point occlusion and light source interference, adjust capture parameters, and improve the accuracy and accuracy of data acquisition.

Benefits of technology

It realizes the accurate division of the quality of optical environment, improves the quality and accuracy of motion capture, optimizes the virtual simulation process, and ensures the reliability and accuracy of data acquisition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118819308B_ABST
    Figure CN118819308B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of virtual simulation technology, and in particular to a virtual simulation method based on self-learning. The method includes step S1, determining the type of the current optical environment; step S2, dividing the third optical environment into a second optical environment or a fourth optical environment; step S3, performing dynamic actions in the second optical environment to perform simulation capture; step S4, comparing the real-time simulation data with the simulation capture data, and correcting the marking point setting parameters according to the comparison results. The present invention analyzes the quality of the capture environment by analyzing the position data of the static target in the capture area, performs motion capture and virtual simulation in an environment with good quality of the current optical environment, and establishes a feedback loop by comparing the simulation of real-time data with the actual capture data to continuously improve the quality and accuracy of motion capture, optimize the current simulation process, and improve the accuracy of the capture task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of virtual simulation, and in particular to a virtual simulation method based on self-learning. Background Art

[0002] Virtual simulation refers to capturing the movements of real data and converting them into the movements of virtual characters to achieve realistic simulation. Virtual simulation methods rely on real-time data obtained by optical motion capture systems, such as position, posture and other information, to generate virtual environments or character movements. Therefore, accurate acquisition of the position of static target markers is crucial for subsequent data analysis. If the markers are blocked, data inaccuracy may occur, affecting subsequent processing and application. Therefore, there is an urgent need for a method to detect and analyze the occlusion and positioning accuracy of markers to improve capture accuracy and reliability, thereby achieving better virtual simulation effects.

[0003] The patent document with Chinese patent authorization announcement number CN114758042B discloses a new virtual simulation engine, virtual simulation method and device, which includes: performing simulation test according to sample data, and determining to receive user instructions when the test result is that the test passes; obtaining data to be drawn according to the user instructions; performing virtual simulation according to the data to be drawn to obtain a simulated image; obtaining sensor data; determining the user's simulated action according to the sensor data; rendering and synchronously displaying according to the simulated action and the simulated image; It can be seen that the existing virtual simulation technology lacks a method of analyzing the quality of the current optical environment to accurately identify and analyze abnormalities in the mark point data, thereby ensuring the accurate positioning of the mark point, and then ensuring the accuracy and effectiveness of data collection, and achieving better virtual simulation effects. Summary of the invention

[0004] To this end, the present invention provides a virtual simulation method based on self-learning to overcome the problem in the prior art that the data collected to generate virtual actions is inaccurate due to the lack of accurate recognition and analysis of marker point data, resulting in poor virtual simulation effect.

[0005] To achieve the above object, the present invention provides a virtual simulation method based on self-learning.

[0006] Obtaining static target images captured by each camera, collecting light source points in each static target image for marking, generating a marking point array, obtaining the total number of marking points in the marking point array for determination, and determining the type of the current optical environment according to the determination result;

[0007] In the third optical environment, respectively obtain the real-time signal strength received by each camera within a preset acquisition time, draw a curve of the real-time signal strength changing over time, obtain a plurality of signal strength change curves, obtain the peak feature of any signal strength change curve for analysis, so as to divide the third optical environment into the second optical environment or the fourth optical environment;

[0008] Under the second optical environment, a dynamic action is performed to perform simulation capture, and real-time position data of the dynamic target is collected in real time during the simulation capture process to capture the target action, and the target action is analyzed and calculated to obtain position data;

[0009] The analysis and calculation process includes preprocessing, removing noise, calculating the angle change of each joint point, identifying the jitter situation in the simulation process, adjusting the capture parameters according to the identification result, and eliminating the jitter situation and forming a simulated action trajectory to obtain the position data;

[0010] generating a real-time simulated action according to the position data, generating simulated capture data according to the simulated action trajectory, comparing the real-time simulated data with the simulated capture data, and modifying the marking point setting parameters according to the comparison result;

[0011] The marking point setting parameter is the standard difference distance.

[0012] Further, obtaining the total number of marking points in the marking point array for determination includes:

[0013] Obtain the total number of marking points in the marking point array to obtain the number of real-time marking points, and determine the number of real-time marking points according to the number of standard marking points;

[0014] When the number of real-time marking points is less than the number of standard marking points, determining that the current optical environment is the first optical environment;

[0015] When the number of real-time marking points is greater than or equal to the number of standard marking points, the real-time marking point data in any of the static target images is obtained and matched with the standard marking point data to analyze whether the current lighting environment is affected by other light sources.

[0016] Furthermore, obtaining the real-time marker point data in any of the static target images and matching the standard marker point data includes:

[0017] Calculate the real-time difference distance between each real-time mark point and the corresponding standard position respectively, and judge each real-time difference distance according to the standard difference distance.

[0018] If the real-time difference distances are all smaller than the standard difference distances, it is determined that the real-time position of the real-time marking point matches the corresponding standard position, and the current optical environment is determined to be the second optical environment;

[0019] If the real-time difference distance is greater than the standard difference distance, the corresponding real-time mark point is marked as a position difference mark point, and the real-time light source brightness of each difference mark point is obtained for analysis to determine whether there is interference in the current optical environment.

[0020] Furthermore, obtaining the real-time light source brightness of each position difference mark point for analysis includes:

[0021] The real-time light source brightness of each position difference mark point is determined according to the first standard light source brightness and the second standard light source brightness.

[0022] If the real-time light source brightness of each position difference mark point is less than or equal to the first standard light source brightness, it is determined that there is no interference, and the current optical environment is determined to be the second optical environment;

[0023] If the real-time light source brightness of a position difference mark point is greater than the first standard light source brightness, it is determined that interference exists, and the corresponding position difference mark point is marked as an entity difference mark point. The shape characteristics of the entity difference mark point are analyzed to determine the degree of interference, so as to determine the quality of the current optical environment according to the degree of interference.

[0024] Further, analyzing the shape features of the entity difference marking points includes:

[0025] Obtain the inner area of ​​the contour of each entity difference mark point, obtain several real-time light source areas, and compare the standard light source area with each real-time light source area.

[0026] If there is a real-time light source area that is smaller than the standard light source area, the corresponding entity difference mark point is marked as a target difference mark point, so as to determine the quality of the current optical environment according to the number of target difference mark points;

[0027] If the areas of each real-time light source are greater than or equal to the standard light source area, it is determined that the current optical environment is the fourth optical environment.

[0028] Further, determining the quality of the current optical environment according to the number of target difference marking points includes:

[0029] Get the percentage of the target difference mark points to the entity difference mark points, get the real-time difference ratio, and compare the real-time difference ratio with the standard difference ratio.

[0030] If the real-time difference ratio is less than the standard difference ratio, the current optical environment is determined to be the third optical environment;

[0031] If the real-time difference ratio is greater than or equal to the standard difference ratio, it is determined that the current optical environment is the fourth optical environment.

[0032] Furthermore, the peak characteristics include real-time peak amplitude and real-time peak number, and obtaining the peak characteristics of any signal strength change curve for analysis includes:

[0033] The signal strength variation curve is corrected according to the real-time peak amplitude of each peak to be corrected into a signal waveform diagram;

[0034] Obtain the number of real-time peaks in the signal waveform diagram, and determine the number of real-time peaks according to the standard number of peaks,

[0035] If the real-time peak number is less than the standard peak number, the current optical environment is determined to be the second optical environment;

[0036] If the number of real-time peaks is greater than or equal to the standard number of peaks, the number of real-time peaks within the next preset acquisition time is obtained for judgment.

[0037] If it is determined again that the real-time peak number is greater than or equal to the standard peak number, the current optical environment is determined to be the fourth optical environment;

[0038] If it is determined that the real-time peak number is less than or equal to the standard peak number, the current optical environment is determined to be the second optical environment.

[0039] Further, the signal strength variation curve is corrected according to the real-time peak amplitude of each peak, including:

[0040] Get the real-time peak amplitude of each peak, and judge the real-time peak amplitude according to the standard peak amplitude.

[0041] If the real-time peak amplitude is less than or equal to the standard peak amplitude, the corresponding peak in the signal strength change curve is deleted;

[0042] If the real-time peak amplitude is greater than the standard peak amplitude, the corresponding peak in the signal strength change curve is not deleted;

[0043] When all the peaks are determined, a signal waveform diagram is obtained.

[0044] Furthermore, the capture parameters include a preset capture frame rate, and adjusting the capture parameters according to the recognition result includes:

[0045] When it is determined that there is jitter, the real-time jitter frequency is obtained, and the real-time jitter frequency is determined according to the standard jitter frequency.

[0046] If the real-time jitter frequency is less than the standard jitter frequency, the preset capture frame rate will not be adjusted;

[0047] If the real-time jitter frequency is greater than or equal to the standard jitter frequency, the preset capture frame rate is adjusted to the corrected capture frame rate.

[0048] Furthermore, the real-time simulation data includes a real-time change curve of the center of mass of the dynamic target on the z-axis, and the simulation capture data includes a virtual change curve of the center of mass of the virtual simulation target on the z-axis. The marking point setting parameters are corrected according to the comparison results, including:

[0049] Calculate the real-time correlation between the real-time change curve and the virtual change curve, and determine the real-time correlation according to the standard correlation.

[0050] If the real-time correlation is less than the standard correlation, the standard difference distance is corrected to the corrected difference distance;

[0051] If the real-time correlation is greater than or equal to the standard correlation, no correction is made to the standard difference distance.

[0052] Compared with the prior art, the beneficial effect of the present invention lies in that the quality of the capture environment is analyzed by analyzing the position data of static targets in the capture area, and the automatic detection of marker point occlusion and light source interference is achieved by analyzing the brightness, shape and relative position. Not only is the current optical environment accurately divided according to the degree of interference, but also the abnormal fluctuations in the infrared signal intensity during the optical motion capture process are effectively identified, and it is determined whether they are temporary or permanent. When temporary interference occurs, it is also determined that the quality of the current optical environment is good. In an environment where the quality of the current optical environment is good, motion capture is performed, virtual simulation is carried out, and a feedback loop is established by comparing the simulation of real-time data with the actual captured data to continuously improve the quality and accuracy of motion capture, optimize the current simulation process, and improve the accuracy of the capture task.

[0053] Furthermore, when it is determined that the number of real-time marker points is less than the standard number of marker points, it means that some marker points are completely blocked, resulting in the number of identified marker points being less than the standard. When it is determined that the number of real-time marker points is greater than or equal to the standard number of marker points, the current optical environment cannot be analyzed based solely on the number of marker points. It is also necessary to consider whether there is interference from other light sources or partial or complete occlusion. If the light source environment is disturbed, it will affect the statistics of the number of marker points and easily cause misjudgment. Moreover, it is impossible to distinguish the degree of occlusion, such as partial occlusion and complete occlusion, by relying solely on changes in the number of marker points. Therefore, it is necessary to further and more carefully analyze the quality of the current optical environment to improve the accuracy of motion capture.

[0054] Furthermore, by continuing to analyze the peak characteristics of the next acquisition cycle to determine the duration of the peak, that is, when it is determined again that the number of real-time peaks is greater than or equal to the number of standard peaks, it means that the peak continues to appear for a long time, and is more likely to be a permanent interference. Therefore, the current optical environment is determined to be the fourth optical environment, and the quality of the current optical environment is accurately identified to ensure the reliability of data acquisition under dynamic conditions and the accuracy of motion capture, thereby achieving better virtual simulation effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of a virtual simulation method based on self-learning according to an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a flow chart of a process for determining the total number of marking points in a marking point array according to an embodiment of the present invention;

[0057] Figure 3 A logic decision diagram for determining the current optical environment according to the peak characteristics according to an embodiment of the present invention;

[0058] Figure 4 A logic decision diagram for adjusting the preset capture frame rate according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0061] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0062] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] See also Figure 1 As shown, it is a flow chart of a virtual simulation method based on self-learning according to an embodiment of the present invention. The present invention provides a virtual simulation method based on self-learning, comprising:

[0064] Step S1, acquiring static target images captured by each camera, collecting light source points in each static target image for marking, generating a marking point array, acquiring the total number of marking points in the marking point array for determination, and determining the type of the current optical environment according to the determination result;

[0065] The types of the current optical environment include a first optical environment, a second optical environment, a third optical environment and a fourth optical environment;

[0066] Step S2, in the third optical environment, respectively obtaining the real-time signal strength received by each camera within a preset acquisition time, drawing a curve of the real-time signal strength changing over time, obtaining a plurality of signal strength changing curves, obtaining a peak feature of any signal strength changing curve for analysis, so as to classify the third optical environment into the second optical environment or the fourth optical environment;

[0067] Step S3, performing a dynamic action in a second optical environment to perform simulation capture, collecting real-time position data of the dynamic target in real time during the simulation capture process to capture the target action, analyzing and calculating the target action to obtain position data;

[0068] The analysis and calculation process includes preprocessing, removing noise, calculating the angle change of each joint point, identifying the jitter situation in the simulation process, adjusting the capture parameters according to the identification result, and eliminating the jitter situation and forming a simulated action trajectory to obtain the position data;

[0069] Step S4, generating a real-time simulated action according to the position data, generating simulated capture data according to the simulated action trajectory, comparing the real-time simulated data with the simulated capture data, and modifying the marking point setting parameters according to the comparison result;

[0070] The simulation process is to simulate the action of the dynamic target;

[0071] The position data in this embodiment is data collected in real time during the motion capture of dynamic targets, including the position changes of the targets in three-dimensional space. Using these data, the system will generate corresponding motion simulations. For example, if a person is walking during the capture process, the position data will reflect his position changes at various time points; by selecting a fixed point in the virtual environment as a reference point, taking this reference point as the origin, extending from the origin to the right horizontally as the x-axis, extending from the origin upward vertically as the y-axis, and extending from the origin to the camera position along the depth direction as the z-axis. The reference point is a specific position in the scene or the center point of the reference object. The range of motion of the captured object or character should be taken into consideration, and it should be ensured that it can cover all the actions that need to be captured. For example, the center position of the sole of the dynamic target is selected as the reference point; motion capture is achieved by performing algorithms such as interpolation or curve fitting on the position data to ensure that continuous and Natural action performance. The goal of this process is to generate a motion model so that subsequent simulation and capture can be based on this model. The simulated motion trajectory is generated based on the real-time simulated motion to express the expected future motion path of the dynamic target. The simulated capture data refers to the data generated on this basis. The data points that may be obtained when actually capturing on this trajectory can be used to predict the position of the target at the next moment, thereby generating a simulated data set, including the motion state and other physical characteristics of the simulated target when it reaches a certain position through numerical calculation, such as speed and acceleration. The real-time simulation data is the data obtained based on the generated motion trajectory, and the simulated capture data is the data obtained during the actual capture process. The two are compared to evaluate the accuracy and reliability of the simulation. The two sets of data are compared by applying the error calculation formula, such as using the Euclidean distance or root mean square error and other evaluation methods to analyze the differences between the simulated values ​​and the real captured values ​​at different time points.

[0072] In this embodiment, the position data of static targets in the capture area are analyzed to analyze the quality of the capture environment. By analyzing the brightness, shape and relative position, automatic detection of marker point occlusion and light source interference is achieved. Not only is the current optical environment accurately divided according to the degree of interference, but also the abnormal fluctuations in the infrared signal intensity during the optical motion capture process are effectively identified, and it is determined whether they are temporary or permanent. When temporary interference occurs, it is also determined that the quality of the current optical environment is good. In an environment where the current optical environment has good quality, motion capture is performed, virtual simulation is carried out, and a feedback loop is established by comparing the simulation of real-time data with the actual captured data to continuously improve the quality and accuracy of motion capture, optimize the current simulation process, and improve the accuracy of the capture task.

[0073] See also Figure 2As shown, it is a schematic flow chart of the process of obtaining the total number of marking points in the marking point array for determination according to an embodiment of the present invention;

[0074] Specifically, in step S1, obtaining the total number of marking points in the marking point array for determination includes:

[0075] Step S101, obtaining the total number of marking points in the marking point array, obtaining the number of real-time marking points, and determining the number of real-time marking points according to the number of standard marking points;

[0076] Step S102, when the number of real-time marking points is less than the number of standard marking points, determining that the current optical environment is the first optical environment;

[0077] Step S103, when the number of real-time marking points is greater than or equal to the number of standard marking points, the real-time marking point data in any of the static target images is obtained and matched with the standard marking point data to analyze whether the current lighting environment is affected by other light sources;

[0078] The number of standard points in this embodiment is the number of sampling points arranged in advance at the dynamic target, that is, the number of marking points pasted in advance at the joint points of the dynamic target, and all marking points form a marking point array.

[0079] When it is determined that the number of real-time marker points is less than the standard number of marker points, it means that some marker points are completely blocked, resulting in the number of identified marker points being less than the standard. When it is determined that the number of real-time marker points is greater than or equal to the standard number of marker points, the current optical environment cannot be analyzed based solely on the number of marker points. It is also necessary to consider whether there is interference from other light sources or partial or complete occlusion. If the light source environment is disturbed, it will affect the statistics of the number of marker points and easily cause misjudgment. Moreover, it is impossible to distinguish the degree of occlusion, such as partial occlusion and complete occlusion, simply relying on changes in the number of marker points. Therefore, it is necessary to further and more carefully analyze the quality of the current optical environment to improve the accuracy of motion capture.

[0080] Specifically, obtaining the real-time marker point data in any of the static target images and matching the standard marker point data includes:

[0081] Calculate the real-time difference distance between each real-time mark point and the corresponding standard position respectively, and judge each real-time difference distance according to the standard difference distance.

[0082] If the real-time difference distances are all smaller than the standard difference distances, it is determined that the real-time position of the real-time marking point matches the corresponding standard position, and the current optical environment is determined to be the second optical environment;

[0083] If the real-time difference distance is greater than the standard difference distance, the corresponding real-time mark point is marked as a position difference mark point, and the real-time light source brightness of each difference mark point is obtained for analysis to determine whether there is interference in the current optical environment.

[0084] The standard difference distance set in this embodiment represents the deviation value between the position data of the captured marking point and the standard position data under static conditions, that is, the allowable marking point position error range. The real-time difference distance is the distance between each real-time captured marking point and the corresponding standard position, which is calculated based on the position coordinates of each real-time marking point and the corresponding standard position coordinates. The set value is related to the capture distance, the size and type of the marking point, etc. The longer the capture distance, the larger the standard difference distance needs to be set. For marking points with smaller sizes or marking points with lower reflective ability, a larger standard difference distance needs to be set. The standard difference distance is generally set in the range of 2 mm to 20 mm, and is selected according to the actual accuracy requirements.

[0085] When it is determined that the real-time difference distance is less than the standard difference distance, it means that in this case, the number of real-time marking points is equal to the number of standard marking points, and the optical environment is not disturbed. When it is determined that the real-time difference distance is greater than the standard difference distance, it means that in this case, the number of real-time marking points is greater than the number of standard marking points. However, even if there are other light sources in the environment, it may not necessarily lead to failure of marking point recognition, because the optical motion capture module uses infrared as the light source, and the influence of other light sources mainly depends on their wavelength and intensity. If the intensity of other light sources is not enough to interfere with the infrared signal, then it will not affect the marking point recognition. The quality of the current optical environment is determined by analyzing the intensity of other light sources.

[0086] Specifically, obtaining the real-time light source brightness of each position difference mark point for analysis includes:

[0087] The real-time light source brightness of each position difference mark point is determined according to the first standard light source brightness and the second standard light source brightness.

[0088] If the real-time light source brightness of each position difference mark point is less than or equal to the first standard light source brightness, it is determined that there is no interference, and the current optical environment is determined to be the second optical environment;

[0089] If the real-time light source brightness of a position difference mark point is greater than the first standard light source brightness, it is determined that interference exists, and the corresponding position difference mark point is marked as an entity difference mark point. The shape characteristics of the entity difference mark point are analyzed to determine the degree of interference, so as to determine the quality of the current optical environment according to the degree of interference.

[0090] The first standard light source brightness set in this embodiment represents the brightness of the infrared light source used by the optical motion capture system under normal working conditions. The second standard light source brightness represents the maximum light source brightness that can be recognized by the optical motion capture system but will not affect the normal operation of the system. By comparing the brightness of the position difference mark point with these two standard brightnesses, it is possible to determine whether there is optical environment interference and the severity of the interference, thereby determining whether motion capture can be performed.

[0091] By obtaining the light source intensity at the difference mark point, when the light source intensity is lower than the standard, it is determined that there is no interference in the current optical environment, and the movement of the dynamic target is executed. That is, when it is determined that the real-time light source brightness of the difference mark points at each position is less than or equal to the first standard light source brightness, it indicates that there are other light sources in the current optical environment, but the intensity of other light sources is not enough to interfere with the infrared signal and can be ignored. When it is determined that the real-time light source brightness of the position difference mark point is greater than the first standard light source brightness, it indicates that there is interference from other light sources. Then, by analyzing the shape characteristics of the interfered mark point, the occlusion degree of the mark point is analyzed to determine whether to perform dynamic target motion capture and simulation in the current optical environment.

[0092] Specifically, analyzing the shape features of the entity difference marking points includes:

[0093] Obtain the inner area of ​​the contour of each entity difference mark point, obtain several real-time light source areas, and compare the standard light source area with each real-time light source area.

[0094] If there is a real-time light source area that is smaller than the standard light source area, the corresponding entity difference mark point is marked as a target difference mark point, so as to determine the quality of the current optical environment according to the number of target difference mark points;

[0095] If the areas of each real-time light source are greater than or equal to the standard light source area, it is determined that the current optical environment is the fourth optical environment.

[0096] The standard light source area in this embodiment represents the area of ​​the mark point irradiated by the infrared light source used by the optical motion capture system under ideal conditions. It is the basis for determining whether the mark point is blocked. If the inner area of ​​the outline of the entity difference mark point is smaller than the standard light source area, the mark point is considered to be blocked and cannot be accurately identified. By comparing the inner area of ​​the outline of the entity difference mark point with the standard light source area, it is possible to determine whether the mark point is blocked, thereby determining whether the current optical environment is suitable for motion capture.

[0097] Specifically, the quality of the current optical environment is determined according to the number of target difference marking points, including:

[0098] Get the percentage of the target difference mark points to the entity difference mark points, get the real-time difference ratio, and compare the real-time difference ratio with the standard difference ratio.

[0099] If the real-time difference ratio is less than the standard difference ratio, the current optical environment is determined to be the third optical environment;

[0100] If the real-time difference ratio is greater than or equal to the standard difference ratio, it is determined that the current optical environment is the fourth optical environment.

[0101] The standard difference ratio in this embodiment represents the ratio of the allowed marker points to be blocked, which is generally set to 5%-10% and is adjusted according to the capture accuracy.

[0102] See also Figure 3 As shown, it is a logical decision diagram for determining the current optical environment according to the peak characteristics according to an embodiment of the present invention;

[0103] Specifically, the peak characteristics include the real-time peak amplitude and the real-time peak number. Acquiring the peak characteristics of any signal strength change curve for analysis includes:

[0104] The signal strength variation curve is corrected according to the real-time peak amplitude of each peak to be corrected into a signal waveform diagram;

[0105] Obtain the number of real-time peaks in the signal waveform diagram, and determine the number of real-time peaks according to the standard number of peaks,

[0106] If the real-time peak number is less than the standard peak number, the current optical environment is determined to be the second optical environment;

[0107] If the number of real-time peaks is greater than or equal to the standard number of peaks, the number of real-time peaks within the next preset acquisition time is obtained for judgment.

[0108] If it is determined again that the real-time peak number is greater than or equal to the standard peak number, the current optical environment is determined to be the fourth optical environment;

[0109] If it is determined that the real-time peak number is less than or equal to the standard peak number, the current optical environment is determined to be the second optical environment;

[0110] The number of real-time peaks in this embodiment represents the frequency of occurrence of each peak within the preset acquisition time, and the standard number of peaks is set to 1. If it is determined that the number of real-time peaks is less than or equal to the standard number of peaks, it means that the frequency of occurrence of peaks is very low. Then, the peak characteristics of the next acquisition cycle are continuously analyzed to determine the duration of the peak. That is, when it is again determined that the number of real-time peaks is greater than or equal to the standard number of peaks, it means that the peak continues to appear for a long time, and is more likely to be permanent interference. Therefore, the current optical environment is determined to be the fourth optical environment, and the quality of the current optical environment is accurately identified to ensure the reliability of data acquisition and the accuracy of motion capture under dynamic conditions, and to achieve better virtual simulation effects.

[0111] Specifically, the signal strength change curve is corrected according to the real-time peak amplitude of each peak, including:

[0112] Get the real-time peak amplitude of each peak, and judge the real-time peak amplitude according to the standard peak amplitude.

[0113] If the real-time peak amplitude is less than or equal to the standard peak amplitude, the corresponding peak in the signal strength change curve is deleted;

[0114] If the real-time peak amplitude is greater than the standard peak amplitude, the corresponding peak in the signal strength change curve is not deleted;

[0115] When all the peaks are determined, a signal waveform diagram is obtained;

[0116] The standard peak amplitude set in this embodiment represents a corresponding lower peak height in the signal strength variation curve, which is used as an indicator for determining the peak type (significant peak or slight fluctuation), and can be set to 2.

[0117] By setting the standard peak amplitude, peaks whose real-time peak amplitude does not exceed the standard peak amplitude are deleted to ignore small fluctuations and focus on significant peaks. When it is determined that the real-time peak amplitude is greater than the standard peak amplitude, it means that a significant peak has appeared. The number of peaks of this type is analyzed to determine whether they are temporary or permanent. This not only enhances the reliability and accuracy of the captured data, but also provides good support for subsequent motion capture.

[0118] In this embodiment, the first optical environment indicates that there is a mark point that is completely blocked, the second optical environment indicates that the optical environment is not disturbed, the third optical environment indicates that the current optical environment is not disturbed to a high degree, which may be temporary or permanent interference, and the fourth optical environment indicates that the current optical environment is highly disturbed, requiring on-site inspection and processing by staff to reposition the camera or the mark point; by conducting in-depth analysis of the changes in the infrared signal intensity of the camera in the third optical environment to identify and analyze abnormal fluctuations, the third optical environment is divided into the second optical environment or the fourth optical environment, and the environmental conditions for motion capture are also met in the case of temporary light source interference.

[0119] See also Figure 4 As shown, it is a logic decision diagram for adjusting the preset capture frame rate according to an embodiment of the present invention;

[0120] Specifically, identifying jitter conditions occurring during the simulation process includes:

[0121] Obtaining historical position data corresponding to several adjacent frames, calculating a displacement standard deviation based on the historical position data, calculating a real-time displacement difference based on position data of adjacent frames and the current frame, comparing the displacement standard deviation with the real-time displacement difference, and determining whether there is jitter based on the comparison result;

[0122] When the real-time displacement difference is smaller than the displacement standard deviation, it is determined that there is no jitter;

[0123] When the real-time displacement difference is greater than or equal to the displacement standard deviation, it is determined that jitter exists;

[0124] Among them, the displacement standard deviation is the standard deviation of the deviation degree of each historical displacement,

[0125] In this embodiment, the displacement standard deviation is first obtained by calculating the average value of the historical position data, then calculating the square of the difference between each historical position data and the average value, adding these square differences and dividing them by the total number of historical position data to obtain the square difference, and taking the square root of the square difference to obtain the displacement standard deviation; the position data represents the coordinate position of the dynamic target in three-dimensional space; the real-time displacement difference represents the amplitude of change of the real-time position, and the real-time displacement difference is obtained by obtaining the position data of the current frame and the previous frame of the current frame, calculating the difference between the position data of these two frames, and obtaining the real-time displacement difference.

[0126] When collecting real-time position data of dynamic targets in real time, the historical position data of the past few frames is obtained through the historical buffer area, and the collected position data is weighted, giving higher weights to the most recent frames of data to smooth out instantaneous large fluctuations caused by jitter.

[0127] Specifically, the capture parameters include a preset capture frame rate, and adjusting the capture parameters according to the recognition result includes:

[0128] When it is determined that there is jitter, the real-time jitter frequency is obtained, and the real-time jitter frequency is determined according to the standard jitter frequency.

[0129] If the real-time jitter frequency is less than the standard jitter frequency, the preset capture frame rate will not be adjusted;

[0130] If the real-time jitter frequency is greater than or equal to the standard jitter frequency, the preset capture frame rate is adjusted to the corrected capture frame rate;

[0131] Wherein, Vc'=max{Vc×[1-(Fs-Fb) / Fs],Vmin}, Vc' is the corrected capture frame rate, Vc is the preset capture frame rate, Fs is the real-time jitter frequency, Fb is the standard jitter frequency, and Vmin is the set minimum capture frame rate.

[0132] The preset capture frame rate in this embodiment represents the number of image frames processed by the system per second during the camera data acquisition process. Generally, the preset capture frame rate is set to 60fps, and is adjusted according to the actual motion capture requirements; the minimum capture frame rate is set to 30fps.

[0133] By analyzing the frequency of jitter, when it is determined that the real-time jitter frequency is greater than or equal to the standard jitter frequency, indicating frequent jitter, the preset capture frame rate is automatically reduced to ensure data quality.

[0134] Specifically, the real-time simulation data includes the real-time change curve of the center of mass of the dynamic target on the z-axis, and the simulation capture data includes the virtual change curve of the center of mass of the virtual simulation target on the z-axis. The correction of the marking point setting parameters according to the comparison results includes:

[0135] Calculate the real-time correlation between the real-time change curve and the virtual change curve, and determine the real-time correlation according to the standard correlation.

[0136] If the real-time correlation is less than the standard correlation, the standard difference distance is corrected to the corrected difference distance;

[0137] If the real-time correlation is greater than or equal to the standard correlation, no correction is made to the standard difference distance;

[0138] Among them, ΔXb'=ΔXb×[1-(Kb-Ks) / Kb], Kb is the standard correlation, Ks is the real-time correlation, ΔXb is the standard difference distance, and ΔXb' is the corrected difference distance.

[0139] In this embodiment, the real-time correlation between the real-time change curve and the virtual change curve is calculated by using the Pearson correlation coefficient. The data of the two curves are collected to ensure that the number of data points is the same, the number of data points is n, the data of the real-time change curve is recorded as x, and the data of the virtual change curve is recorded as y. By calculating the mean of x and the mean of y, the covariance of x and y is calculated, and the standard deviation of x and y is calculated respectively, so as to calculate the real-time correlation: Rs= ;

[0140] Among them, Rs is the real-time correlation, covx,y is the covariance of x and y, Ax is the standard deviation of x, and Ay is the standard deviation of y.

[0141] The standard correlation in this embodiment is set to 80%.

[0142] The virtual simulation method provided in this embodiment can be applied in multiple scenarios. For example, it can be used in ideological and political course education to simulate practical operations and help students practice and improve relevant skills in a safe environment. It can also be used for case analysis to help students better understand and analyze the moral and legal issues in the case by simulating the behavior and expressions of real people.

[0143] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A virtual simulation method based on self-learning, It is characterized in that include, Obtaining static target images captured by each camera, collecting light source points in each static target image for marking, generating a marking point array, obtaining the total number of marking points in the marking point array for determination, and determining the type of the current optical environment according to the determination result; In the third optical environment, respectively obtain the real-time signal strength received by each camera within a preset acquisition time, draw a curve of the real-time signal strength changing over time, obtain a plurality of signal strength change curves, obtain the peak feature of any signal strength change curve for analysis, so as to divide the third optical environment into the second optical environment or the fourth optical environment; Under the second optical environment, a dynamic action is performed to perform simulation capture, and real-time position data of the dynamic target is collected in real time during the simulation capture process to capture the target action, and the target action is analyzed and calculated to obtain position data; The analysis and calculation process includes preprocessing, removing noise, calculating the angle change of each joint point, identifying the jitter in the simulation process, adjusting the capture parameters according to the identification results, and eliminating the jitter and forming a simulated motion trajectory to obtain the position data; generating a real-time simulated action according to the position data, generating simulated capture data according to the simulated action trajectory, comparing the real-time simulated data with the simulated capture data, and modifying the marking point setting parameters according to the comparison result; The marking point setting parameter is the standard difference distance; Wherein, obtaining the total number of marking points in the marking point array for determination includes: Obtain the total number of marking points in the marking point array to obtain the number of real-time marking points, and determine the number of real-time marking points according to the number of standard marking points; When the number of real-time marking points is less than the number of standard marking points, determining that the current optical environment is the first optical environment; When the number of real-time marking points is greater than or equal to the number of standard marking points, the real-time marking point data in any of the static target images is obtained and matched with the standard marking point data to analyze whether the current lighting environment is affected by other light sources; Acquiring the real-time marker point data in any of the static target images and matching the standard marker point data comprises: Calculate the real-time difference distance between each real-time mark point and the corresponding standard position respectively, and judge each real-time difference distance according to the standard difference distance. If the real-time difference distances are all smaller than the standard difference distances, it is determined that the real-time position of the real-time marking point matches the corresponding standard position, and the current optical environment is determined to be the second optical environment; If there is a real-time difference distance greater than the standard difference distance, the corresponding real-time mark point is marked as a position difference mark point, and the real-time light source brightness of each difference mark point is obtained for analysis to determine whether there is interference in the current optical environment; Obtain the real-time light source brightness of each position difference mark point for analysis, including: The real-time light source brightness of each position difference mark point is determined according to the first standard light source brightness and the second standard light source brightness. If the real-time light source brightness of each position difference mark point is less than or equal to the first standard light source brightness, it is determined that there is no interference, and the current optical environment is determined to be the second optical environment; If the real-time light source brightness of the position difference mark point is greater than the first standard light source brightness, it is determined that interference exists, and the corresponding position difference mark point is marked as an entity difference mark point, and the shape characteristics of the entity difference mark point are analyzed to determine the degree of interference, so as to determine the quality of the current optical environment according to the degree of interference; Analyzing the shape features of the entity difference marking points includes: Obtain the inner area of ​​the contour of each entity difference mark point, obtain several real-time light source areas, and compare the standard light source area with each real-time light source area. If there is a real-time light source area that is smaller than the standard light source area, the corresponding entity difference mark point is marked as a target difference mark point, so as to determine the quality of the current optical environment according to the number of target difference mark points; If the areas of each real-time light source are greater than or equal to the standard light source area, it is determined that the current optical environment is the fourth optical environment; The quality of the current optical environment is determined based on the number of target difference markers, including: Get the percentage of the target difference mark points to the entity difference mark points, get the real-time difference ratio, and compare the real-time difference ratio with the standard difference ratio. If the real-time difference ratio is less than the standard difference ratio, the current optical environment is determined to be the third optical environment; If the real-time difference ratio is greater than or equal to the standard difference ratio, the current optical environment is determined to be the fourth optical environment; The peak characteristics include real-time peak amplitude and real-time peak number. Acquiring the peak characteristics of any signal strength change curve for analysis includes: The signal strength variation curve is corrected according to the real-time peak amplitude of each peak to be corrected into a signal waveform diagram; Obtain the number of real-time peaks in the signal waveform diagram, and determine the number of real-time peaks according to the standard number of peaks, If the real-time peak number is less than the standard peak number, the current optical environment is determined to be the second optical environment; If the number of real-time peaks is greater than or equal to the standard number of peaks, the number of real-time peaks within the next preset acquisition time is obtained for judgment. If it is determined again that the real-time peak number is greater than or equal to the standard peak number, the current optical environment is determined to be the fourth optical environment; If it is determined that the real-time peak number is less than or equal to the standard peak number, the current optical environment is determined to be the second optical environment.

2. The virtual simulation method based on self-learning according to claim 1, It is characterized in that Correction of the signal strength change curve according to the real-time peak amplitude of each peak includes: Get the real-time peak amplitude of each peak, and judge the real-time peak amplitude according to the standard peak amplitude. If the real-time peak amplitude is less than or equal to the standard peak amplitude, the corresponding peak in the signal strength change curve is deleted; If the real-time peak amplitude is greater than the standard peak amplitude, the corresponding peak in the signal strength change curve is not deleted; When all the peaks are determined, a signal waveform diagram is obtained.

3. The virtual simulation method based on self-learning according to claim 1, It is characterized in that The capture parameters include a preset capture frame rate, and adjusting the capture parameters according to the recognition result includes: When it is determined that there is jitter, the real-time jitter frequency is obtained, and the real-time jitter frequency is determined according to the standard jitter frequency. If the real-time jitter frequency is less than the standard jitter frequency, the preset capture frame rate will not be adjusted; If the real-time jitter frequency is greater than or equal to the standard jitter frequency, the preset capture frame rate is adjusted to the corrected capture frame rate.

4. The virtual simulation method based on self-learning according to claim 1, It is characterized in that The real-time simulation data includes the real-time change curve of the center of mass of the dynamic target on the z-axis, and the simulation capture data includes the virtual change curve of the center of mass of the virtual simulation target on the z-axis. The parameters of the marking point setting are corrected according to the comparison results, including: Calculate the real-time correlation between the real-time change curve and the virtual change curve, and determine the real-time correlation according to the standard correlation. If the real-time correlation is less than the standard correlation, the standard difference distance is corrected to the corrected difference distance; If the real-time correlation is greater than or equal to the standard correlation, no correction is made to the standard difference distance.

Citation Information

Patent Citations

  • New virtual simulation engines, virtual simulation methods, and devices

    CN114758042B

  • Example learning system and method of robot assembly off-line for motion capture

    CN110561450A

  • Optical motion capture data processing method, model training method and device

    CN113688683A