Motion capture camera reference method, system, terminal and storage medium

Abstract

The present invention relates to a motion capture camera reference method, system, terminal and storage medium, and relates to the field of motion capture. The present invention is used to obtain reference image parameters through a reference camera, and compare the reference image parameters with the motion capture feature parameters captured by the motion capture camera according to time parameters, so as to analyze and locate abnormal feature parameters in the motion capture feature parameters. The present application has the effect of improving the efficiency of motion capture and improving the analysis speed of feature information captured by the motion capture camera.

Description

Technical Field

[0001] The present invention relates to the field of motion capture, and in particular to a motion capture camera reference method, system, terminal and storage medium. Background Art

[0002] A motion capture camera is a high-precision optical camera used to capture the motion of an object.

[0003] In the prior art, motion capture generally first captures the feature information of the action through a motion capture camera, and then models the feature information through a processing terminal to restore the action. Existing motion capture cameras generally only capture black and white images, and extract feature information from the black and white images for uploading.

[0004] When there is a problem with the modeling, it is not easy to locate the erroneous feature information by analyzing the feature information provided by the motion capture camera, which leads to a decrease in the efficiency of motion capture. Summary of the invention

[0005] In order to improve the efficiency of motion capture and increase the speed of analyzing feature information captured by a motion capture camera, the present invention provides a motion capture camera reference method, system, terminal and storage medium.

[0006] In a first aspect, the present invention provides a motion capture camera reference method, system, terminal and storage medium, which adopt the following technical solutions:

[0007] A motion capture camera reference method, comprising:

[0008] Controlling a preset motion capture camera to obtain motion capture feature parameters and motion capture time parameters, and controlling a preset reference camera to obtain reference image parameters and reference time parameters;

[0009] Matching a motion model from a preset motion model building model according to motion capture feature parameters;

[0010] Judging whether there is a model abnormality from a preset model abnormality judgment model according to the motion model;

[0011] When there is a model abnormality, mapping image parameters and mapping time parameters are matched from a preset image mapping model according to the motion capture feature parameters, the motion capture time parameters, the reference image parameters and the reference time parameters;

[0012] identifying abnormal time parameters from a preset action abnormality recognition model according to the mapped image parameters and the mapped time parameters;

[0013] A removal feature parameter is obtained from a preset anomaly removal model according to an anomaly time parameter, a motion capture time parameter and a motion capture feature parameter.

[0014] By adopting the above technical solution, when the motion model established by the motion capture feature parameters has an abnormality, the motion capture feature parameters and the reference image parameters are combined according to the time parameters to form a mapping data set, so as to locate the time period when the abnormal parameters appear by analyzing the motion capture features and the reference image in the mapping data set, and then remove the abnormal parameters from the motion capture feature parameters, thereby improving the analysis speed of the feature information captured by the motion capture camera and improving the efficiency of motion capture.

[0015] Optionally, an image display method is also included, and the image display method includes:

[0016] Select the required resolution according to the preset resolution determination method;

[0017] Matching a reference frame rate from a preset frame rate matching model according to the required resolution;

[0018] Matching a display image from a preset display image matching model according to a reference frame rate, a required resolution and reference image parameters;

[0019] Control the preset display device to display the display image.

[0020] By adopting the above technical solution, a suitable output frame rate is matched according to the selected output resolution, and the reference image parameters are adapted to a clear image according to the output resolution and the output frame rate, so that the image is output through a display device to preview the scene.

[0021] Optionally, the image display method further includes:

[0022] Selecting a required frame rate according to a preset frame rate determination method;

[0023] Match the reference resolution from the preset resolution matching model according to the required frame rate;

[0024] Matching a display image from a preset display image matching model according to a reference resolution, a required frame rate, and reference image parameters;

[0025] Control the preset display device to display the display image.

[0026] By adopting the above technical solution, a suitable output resolution is matched according to the selected output frame rate, and the reference image parameters are adapted to a clear image according to the output resolution and the output frame rate, so that the image is output through a display device to preview the scene.

[0027] Optionally, a feature tracking method is also included, and the feature tracking method includes:

[0028] Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to the reference image parameters;

[0029] When there is a motion feature in the picture, a reference offset is calculated from a preset reference offset calculation model based on the motion feature;

[0030] When the reference offset exceeds the preset offset range, matching the rotation angle from a preset rotation angle matching model according to the reference offset;

[0031] Matching the rotation angle from a preset rotation angle matching model according to the feature distance and the reference offset;

[0032] Controls the preset reference camera rotation according to the rotation angle.

[0033] By adopting the above technical solution, the reference offset is obtained by identifying the distance of the motion feature from the center of the image in the reference image, and when the reference offset is too large, the reference camera is controlled to rotate to reduce the excessive movement amplitude of the motion feature, causing the motion feature to escape from the capture range of the reference camera, thereby resulting in incomplete motion capture of the motion feature.

[0034] Optionally, the feature tracking method further includes:

[0035] When there are motion features in the picture, the feature position is matched from a preset feature position matching model based on the motion features, and the motion capture angle and motion capture position of the motion capture camera are obtained;

[0036] Calculating a motion capture offset from a preset motion capture offset calculation model according to the feature position, the motion capture angle, and the motion capture position;

[0037] When the motion capture offset exceeds the preset capture interval, the capture angle is matched from the preset capture angle matching model according to the feature position and the motion capture position;

[0038] Calculate the difference between the capture angle and the motion capture angle and define it as the motion capture angle;

[0039] Control the rotation of a preset motion capture camera according to the motion capture angle, and match the motion capture distance from a preset motion capture distance matching model according to the feature position and the motion capture position;

[0040] When the motion capture distance exceeds the preset capture interval, the feature 3D is matched from the preset feature 3D recognition model based on the motion features;

[0041] Matching the capture distance from a preset capture distance matching model according to the three-dimensional feature;

[0042] Calculate the difference between the capture distance and the motion capture distance and define it as the moving distance;

[0043] Matching the moving stroke from a preset moving stroke matching model according to the moving distance and the capture angle;

[0044] Control the movement of the vehicle preset under the motion capture camera according to the movement stroke.

[0045] By adopting the above technical solution, the motion capture offset of the motion feature in the motion capture feature parameters is determined according to the relative position of the motion feature and the motion capture camera, so that when the motion capture offset is too large, the motion capture camera is controlled to rotate to reduce the situation where the motion feature escapes from the capture range of the motion capture camera, and the distance between the motion capture camera and the motion feature is controlled according to the size of the motion capture feature, so that the motion capture camera can capture the entire motion feature.

[0046] Optionally, a standby control method is also included, and the standby control method includes:

[0047] When there is no motion feature in the picture, the preset reference camera is controlled to obtain a standby image according to the preset standby frequency, and the preset motion capture camera is controlled to stop obtaining parameters;

[0048] Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to the standby image;

[0049] When there are motion features in the picture, the capture frequency is matched from a preset capture frequency matching model based on the motion features;

[0050] Controlling a preset reference camera to obtain captured image parameters and reference time parameters according to the capture frequency;

[0051] According to the captured image parameters, reference image parameters are matched from a preset reference image optimization model, and a preset motion capture camera is controlled to obtain motion capture feature parameters and motion capture time parameters.

[0052] By adopting the above technical solution, when there is no motion feature, the energy consumption of the motion capture system is reduced by reducing the capture frequency of the reference image and turning off the motion capture camera, and the motion capture camera is started when the reference camera recognizes the motion feature. At the same time, the frequency of the reference camera capturing the reference image is matched according to the changing speed of the motion feature, and the image captured by the reference camera is compressed and interpolated to form clear and continuous reference image parameters.

[0053] Optionally, a characteristic parameter optimization method is also included, and the characteristic parameter optimization method includes:

[0054] Matching the contrast image parameters from a preset contrast image matching model according to the reference image parameters, the reference time parameters and the abnormal time parameters;

[0055] Extracting reference feature parameters from a preset reference feature extraction model according to the comparison image parameters;

[0056] Matching optimized feature parameters from a preset feature parameter optimization model according to reference feature parameters, abnormal time parameters, motion capture feature parameters and motion capture time parameters;

[0057] The merged feature parameters are matched from the preset feature merging model according to the optimized feature parameters, the abnormal time parameters and the removed feature parameters.

[0058] By adopting the above technical solution, the feature parameters of the removed part of the feature parameters are re-extracted by referring to the image parameters, and the feature parameters are compared with the abnormal parameters in the motion capture feature parameters to optimize the feature parameters, and the optimized feature parameters are filled into the vacant parts of the removed feature parameters to obtain complete feature parameters.

[0059] In a second aspect, the present application provides a motion capture camera reference system, which adopts the following technical solution:

[0060] A motion capture camera reference system, comprising:

[0061] An acquisition module, used to acquire motion capture feature parameters, motion capture time parameters, reference image parameters, reference time parameters, motion capture angles, motion capture positions, standby images, and captured image parameters;

[0062] A memory, used to store a program of a control method of any of the above-mentioned motion capture camera reference methods;

[0063] The program in the memory can be loaded and executed by the processor to implement any of the above-mentioned control methods of the motion capture camera reference method.

[0064] In a third aspect, the present application provides a computer storage medium capable of storing a corresponding program, which is convenient for implementing a motion capture camera reference method, and adopts the following technical solution:

[0065] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the above-mentioned motion capture camera reference methods.

[0066] In a fourth aspect, the present application provides an intelligent terminal, which adopts the following technical solution:

[0067] An intelligent terminal, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute any one of the above-mentioned motion capture camera reference methods.

[0068] By adopting the above technical solution, when the motion model established by the motion capture feature parameters has an abnormality, the motion capture feature parameters and the reference image parameters are combined according to the time parameters to form a mapping data set, so as to locate the time period when the abnormal parameters appear by analyzing the motion capture features and the reference image in the mapping data set, and then remove the abnormal parameters from the motion capture feature parameters, thereby improving the analysis speed of the feature information captured by the motion capture camera and improving the efficiency of motion capture.

[0069] In summary, the present application includes at least one of the following beneficial technical effects:

[0070] 1. When the motion model established by the motion capture feature parameters has an abnormality, the motion capture feature parameters and the reference image parameters are combined according to the time parameters to form a mapping data set, so as to locate the time period when the abnormal parameters appear by analyzing the motion capture features and reference images in the mapping data set, and then remove the abnormal parameters from the motion capture feature parameters, thereby improving the analysis speed of the feature information captured by the motion capture camera and improving the efficiency of motion capture;

[0071] 2. Matching a suitable output frame rate according to the selected output resolution, and adapting the reference image parameters to a clear image according to the output resolution and the output frame rate, so as to output the image through a display device to preview the scene;

[0072] 3. Determine the motion capture offset of the motion feature in the motion capture feature parameters according to the relative position of the motion feature and the motion capture camera, so as to control the rotation of the motion capture camera when the motion capture offset is too large to reduce the situation where the motion feature is out of the capture range of the motion capture camera, and control the distance between the motion capture camera and the motion feature according to the size of the motion capture feature, so that the motion capture camera can capture the entire motion feature. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a flow chart of a motion capture camera reference method;

[0074] Figure 2 This is the process of the image display method Figure 1 ;

[0075] Figure 3 This is the process of the image display method Figure 2 ;

[0076] Figure 4 This is the process of feature tracking method Figure 1 ;

[0077] Figure 5 This is the process of feature tracking method Figure 2 ;

[0078] Figure 6 This is the process of feature tracking method Figure 3 ;

[0079] Figure 7 is a flow chart of a standby control method;

[0080] Figure 8 It is a flow chart of the feature parameter optimization method. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following Figure 1 - Attached 8 and embodiments further illustrate the present invention in detail. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0082] The embodiments of the present application disclose a motion capture camera reference method, system, terminal and storage medium. The present invention is used to obtain reference image parameters through a reference camera, and compare the reference image parameters with motion capture feature parameters captured by the motion capture camera according to time parameters, so as to analyze and locate abnormal feature parameters in the motion capture feature parameters.

[0083] Reference Figure 1 , a motion capture camera reference method, comprising the following steps:

[0084] Step 100: controlling a preset motion capture camera to obtain motion capture feature parameters and motion capture time parameters, and controlling a preset reference camera to obtain reference image parameters and reference time parameters.

[0085] The motion capture camera refers to a device used to extract feature points in a black and white image by acquiring a black and white image, thereby forming motion capture feature parameters. The reference camera refers to a device used to acquire a color high frame rate image to form reference image parameters. The reference camera may be a CMOS Sensor of model SC132GS, which has a full frame rate of 120fps and an output image in YUV420 format.

[0086] The image directly output by SC132GS is 1.3M per frame, which is 1.3x120=156M per second at 120 frames per second. The data volume is large and is not conducive to network transmission. Therefore, the original image needs to be encoded to reduce its size. Generally, the hardware encoder is used to encode it in H264. The compression ratio before and after encoding is 160:1, and the data is 0.9M per second. The motion capture camera and reference camera are selected by the staff according to the actual situation, so I will not go into details here.

[0087] The motion capture time parameters are the time information corresponding to the motion capture feature parameters, and the reference time parameters are the time information corresponding to the reference image parameters. The reference camera and the motion capture camera use the ptpv1 protocol to achieve time synchronization with a synchronization accuracy of milliseconds. The method for obtaining the motion capture time parameters and the reference time parameters is selected by the staff based on actual conditions and will not be elaborated here.

[0088] Step 101: Match a motion model from a preset motion model building model according to motion capture feature parameters.

[0089] The motion model building model refers to a neural network model obtained through sample training. The motion model building model can restore the motion model of the source of the motion capture feature parameters according to the motion capture feature parameters.

[0090] Step 102: Determine whether there is a model abnormality from a preset model abnormality determination model according to the motion model.

[0091] The model anomaly judgment model refers to a neural network model obtained through sample training. The model anomaly judgment model can judge whether model anomalies such as jitter occur in the motion model.

[0092] Step 103: When there is a model abnormality, mapping image parameters and mapping time parameters are matched from a preset image mapping model according to the motion capture feature parameters, the motion capture time parameters, the reference image parameters and the reference time parameters.

[0093] The image mapping model refers to a neural network model obtained through sample training. The image mapping model can map the reference image parameters to the motion capture feature parameters according to the motion capture time parameters and the reference time parameters, so that the reference image parameters and the motion capture feature parameters of each frame correspond to each other to form mapping image parameters and mapping time parameters.

[0094] Step 104: identifying abnormal time parameters from a preset action abnormality recognition model according to the mapped image parameters and the mapped time parameters.

[0095] The motion anomaly recognition model refers to a neural network model obtained through sample training. The motion anomaly recognition model can compare the reference image and motion capture features of each frame in the mapped image parameters to identify the abnormal time parameters corresponding to the abnormal location.

[0096] Step 105: Obtain removal feature parameters from a preset anomaly removal model according to the anomaly time parameter, the motion capture time parameter and the motion capture feature parameter.

[0097] The anomaly removal model refers to a neural network model obtained through sample training. The anomaly removal model can delete the feature parameters corresponding to the abnormal time parameters in the motion capture feature parameters to reduce the abnormality of the motion model and facilitate the subsequent re-collection of the motion to form a complete motion model.

[0098] refer to Figure 2 , the image display method comprises the following steps:

[0099] Step 200: Select the required resolution according to a preset resolution determination method.

[0100] The resolution determination method refers to a method for selecting the required resolution of an image for previewing a scene, and is generally a method of manual selection by staff. The resolution determination method is selected by staff based on actual conditions and will not be elaborated here.

[0101] Step 201: Match a reference frame rate from a preset frame rate matching model according to the required resolution.

[0102] The frame rate matching model refers to a neural network model obtained through sample training. The frame rate matching model can match the reference frame rate corresponding to the required resolution. When the total data volume remains unchanged, the frame rate can be increased by reducing the resolution. In this embodiment, the frame rate of SC132GS at full resolution 1280x1072 is up to 120 frames. When the resolution is reduced, the frame rate can be increased, which are 1280x720 (120fps), 960x540 (150fps), 640x480 (220).

[0103] Step 202: Match a display image from a preset display image matching model according to a reference frame rate, a required resolution and reference image parameters.

[0104] The display image matching model refers to a neural network model obtained through sample training. The display image matching model can convert reference image parameters into a display image according to a reference frame rate and required resolution.

[0105] Step 203: Control a preset display device to display an image.

[0106] The display device refers to a device used to preview the captured scene. The display device is generally set on the host computer. The reference camera generally communicates with the host computer through the GigE protocol. The display device is selected by the staff according to the actual situation and will not be elaborated here.

[0107] refer to Figure 3 , the image display method further comprises the following steps:

[0108] Step 204: Select the required frame rate according to a preset frame rate determination method.

[0109] The frame rate determination method refers to a method for selecting the required frame rate of an image used to preview a scene, and is generally a method of manual selection by staff. The frame rate determination method is selected by staff based on actual conditions and will not be elaborated here.

[0110] Step 205: Match a reference resolution from a preset resolution matching model according to the required frame rate.

[0111] The resolution matching model refers to a neural network model obtained through sample training. The resolution matching model can match the reference resolution corresponding to the required frame rate. When the total data volume remains unchanged, reducing the frame rate can improve its resolution.

[0112] Step 206: Match a display image from a preset display image matching model according to the reference resolution, the required frame rate and the reference image parameters.

[0113] The display image is an image formed by converting the reference image parameters according to the reference resolution and the required frame rate through the display image matching model.

[0114] Step 207: Control a preset display device to display an image.

[0115] The scene captured by the reference camera is displayed through a display device, so that the staff can adjust the setting positions of the reference camera and the motion capture camera conveniently so that the reference camera and the motion capture camera can completely capture the target.

[0116] refer to Figure 4 ,The feature tracking method includes the following steps:

[0117] Step 300: Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to reference image parameters.

[0118] The motion feature refers to the object to be motion captured. The motion feature recognition model refers to a neural network model obtained through sample training, and the motion feature recognition model can recognize whether there is a motion feature in the reference image parameters.

[0119] Step 301: When there is a motion feature in the picture, a reference offset is calculated from a preset reference offset calculation model based on the motion feature.

[0120] The reference offset calculation model refers to a neural network model obtained through sample training. The reference offset calculation model can calculate the distance between the center of the motion feature in the reference image parameters and the center of the reference image, and define the distance as the reference offset.

[0121] Step 302: When the reference offset exceeds a preset offset interval, a rotation angle is matched from a preset rotation angle matching model according to the reference offset.

[0122] The offset interval refers to the interval of the distance between the center of the motion feature and the center of the reference image. The offset interval is selected by the staff according to the actual situation and will not be described here. If the reference offset exceeds the offset interval, it means that the center of the motion feature is too far away from the center of the reference image, which may easily cause the motion feature to escape the capture range of the reference camera. The rotation angle matching model refers to the neural network model obtained through sample training. The rotation angle matching model can match the rotation angle that the reference camera needs to rotate to make the center of the motion feature return to the center of the reference image.

[0123] Step 303: Control the rotation of a preset reference camera according to the rotation angle.

[0124] When the motion feature deviates too much from the center of the reference image, the reference camera is rotated to bring the center of the motion feature back to the center of the reference image, thereby reducing the situation where the motion feature exceeds the capture range of the reference camera.

[0125] refer to Figure 5 and Figure 6 , the feature tracking method also includes the following steps:

[0126] Step 304: When there is a motion feature in the picture, the feature position is matched from a preset feature position matching model based on the motion feature, and the motion capture angle and motion capture position of the motion capture camera are obtained.

[0127] The feature position matching model refers to a neural network model obtained through sample training. The feature position matching model can identify the feature position of the motion feature in the reference image parameters. The motion capture angle refers to the angle of the motion capture camera, and the motion capture position refers to the position of the motion capture camera. The motion capture angle can be obtained by the angle sensor set on the motion capture camera. The motion capture position can be obtained by obtaining the image of the motion capture camera and then identifying it from the image. The method of obtaining the motion capture angle and motion capture position is selected by the staff according to the actual situation and will not be elaborated here.

[0128] Step 305: Calculate the motion capture offset from a preset motion capture offset calculation model according to the feature position, the motion capture angle and the motion capture position.

[0129] The motion capture offset calculation model refers to a neural network model obtained through sample training. The motion capture offset calculation model can calculate the distance between the center of the motion feature and the center of the image obtained by the motion capture camera under the conditions of feature position, motion capture angle and motion capture position, and define the distance as the motion capture offset.

[0130] Step 306: When the motion capture offset exceeds the preset capture interval, a capture angle is matched from a preset capture angle matching model according to the feature position and the motion capture position.

[0131] The capture interval refers to the interval of the distance between the center of the motion feature and the center of the image captured by the motion capture camera. The capture interval is selected by the staff according to the actual situation and will not be described here. The motion capture offset exceeds the capture interval, which means that the center of the motion feature is too far away from the center captured by the motion capture camera, which may easily cause the motion feature to escape the capture range of the motion capture camera.

[0132] The capture angle matching model refers to a neural network model obtained through sample training. The capture angle matching model can match the capture angle of the motion capture camera when the center of the motion feature is located at the center of the picture obtained by the motion capture camera.

[0133] Step 307: Calculate the difference between the capture angle and the motion capture angle, and define it as the motion capture rotation angle.

[0134] The motion capture angle refers to the angle that the motion capture camera needs to rotate from the motion capture angle to the capture angle.

[0135] Step 308: controlling the preset motion capture camera to rotate according to the motion capture rotation angle, and matching the motion capture distance from a preset motion capture distance matching model according to the feature position and the motion capture position.

[0136] When the motion feature deviates too much from the capture center of the motion capture camera, the motion capture camera is rotated to bring the center of the motion feature back to the capture center of the motion capture camera, thereby reducing the situation where the motion feature exceeds the capture range of the motion capture camera.

[0137] The motion capture distance matching model refers to a neural network model obtained through sample training. The motion capture distance matching model can calculate the motion capture distance between the feature position and the motion capture position.

[0138] Step 309: When the motion capture distance exceeds the preset capture interval, the feature three-dimensional is matched from a preset feature three-dimensional recognition model based on the motion feature.

[0139] The capture interval refers to the distance interval in which the motion capture camera can obtain clear images and thus obtain accurate feature parameters. The capture interval is selected by the staff according to the actual situation and will not be described here. If the motion capture distance exceeds the capture interval, it means that the motion capture camera is too far away from the motion feature and it is not easy for the motion capture camera to obtain accurate feature parameters.

[0140] The feature three-dimensional recognition model refers to a neural network model obtained through sample training. The feature three-dimensional recognition model can match the feature three-dimensional of motion features from reference image parameters.

[0141] Step 310: Match the capture distance from a preset capture distance matching model according to the three-dimensional feature.

[0142] The capture distance matching model refers to a neural network model obtained through sample training. The capture distance matching model can match the minimum capture distance for the motion capture camera to capture the complete motion feature when the motion feature is located at the capture center of the motion capture camera.

[0143] Step 311: Calculate the difference between the capture distance and the motion capture distance, and define it as the moving distance.

[0144] The moving distance refers to the moving distance that the motion capture camera needs to move when adjusting the distance between the motion capture camera and the motion feature from the motion capture distance to the capture distance.

[0145] Step 312: Match the moving stroke from a preset moving stroke matching model according to the moving distance and the capture angle.

[0146] The movement stroke matching model refers to a neural network model obtained through sample training. The movement stroke matching model can match the movement stroke that causes the motion capture camera to move according to the moving distance and capture angle.

[0147] Step 313: Control the movement of the carrier preset under the motion capture camera according to the movement stroke.

[0148] The carrier is a device used to drive the motion capture camera to move. The carrier is selected by the staff according to the actual situation and will not be described in detail here. When the distance between the motion feature and the motion capture camera is too large, the motion feature is tracked by the carrier, thereby reducing the abnormal motion capture feature parameters obtained by the motion capture camera.

[0149] Reference Figure 7 , the standby control method comprises the following steps:

[0150] Step 400: When there is no motion feature in the picture, the preset reference camera is controlled to obtain a standby image according to the preset standby frequency, and the preset motion capture camera is controlled to stop obtaining parameters.

[0151] The standby frequency refers to the frequency of acquiring images used by the reference camera to save energy. The standby frequency is selected by the staff according to the actual situation and will not be described in detail here. The standby image is the image acquired by the reference camera according to the standby frequency.

[0152] Step 401: Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to the standby image.

[0153] The motion feature recognition model is used to identify whether there is a motion feature in the standby image.

[0154] Step 402: When there is a motion feature in the picture, a capture frequency is matched from a preset capture frequency matching model based on the motion feature.

[0155] The capture frequency matching model refers to a neural network model obtained through sample training. The capture frequency matching model can match the capture frequency of the motion changes of the motion features fully displayed in the pictures captured by the reference camera. When the motion changes of the motion features are slow, a lower capture frequency is used. When the motion changes of the motion features are fast, a higher capture frequency is used, thereby improving the energy utilization of the reference camera.

[0156] Step 403: Control a preset reference camera to obtain captured image parameters and reference time parameters according to the capture frequency.

[0157] The reference camera obtains pictures of motion features, which serve as reference pictures for the motion capture camera.

[0158] Step 404: matching reference image parameters from a preset reference image optimization model according to the captured image parameters, and controlling a preset motion capture camera to obtain motion capture feature parameters and motion capture time parameters.

[0159] The reference image optimization model refers to a neural network model obtained through sample training. The reference image optimization model can optimize the captured image parameters to form the final reference image parameters. The image optimization process generally includes transferring the image output by the reference camera into the hardware ISP in the form of a data stream for processing (image white balance calibration, automatic exposure, automatic gain, dark level calibration, etc.); the optimization process also includes when a lower capture frequency is used to obtain a captured image, the captured image parameters are formed into complete and continuous reference image parameters through operations such as frame interpolation.

[0160] refer to Figure 8 , the feature parameter optimization method includes the following steps:

[0161] Step 500: matching the contrast image parameters from a preset contrast image matching model according to the reference image parameters, the reference time parameters and the abnormal time parameters.

[0162] The contrast image matching model refers to a neural network model obtained through sample training. The contrast image matching model can extract image parameters between abnormal time parameters in the reference image parameters to form contrast image parameters.

[0163] Step 501: extracting reference feature parameters from a preset reference feature extraction model according to the contrast image parameters.

[0164] The reference feature extraction model refers to a neural network model obtained through sample training. The reference feature extraction model can extract feature points from contrast image parameters to form reference feature parameters that are consistent with the motion capture feature parameter format.

[0165] Step 502: Matching optimized feature parameters from a preset feature parameter optimization model according to reference feature parameters, abnormal time parameters, motion capture feature parameters and motion capture time parameters.

[0166] The feature parameter optimization model refers to a neural network model obtained through sample training. The feature parameter optimization model can combine the feature parameters located in the abnormal time parameter segment in the motion capture feature parameters with the reference feature parameters to form optimized feature parameters. The combination process includes retaining the same feature points in the motion capture feature parameters and the reference feature parameters, removing the different feature points in the motion capture feature parameters and the reference feature parameters, and then supplementing the feature points according to the changes in the feature points before and after the motion capture feature parameters.

[0167] Step 503: Match the merged feature parameters from the preset feature merging model according to the optimized feature parameters, the abnormal time parameters and the removed feature parameters.

[0168] The feature merging model refers to a neural network model obtained through sample training. The feature merging model can insert the optimized feature parameters into the removed feature parameters according to the abnormal time parameters to form merged feature parameters.

[0169] Based on the same inventive concept, an embodiment of the present invention provides a motion capture camera reference system, including:

[0170] An acquisition module, used to acquire motion capture feature parameters, motion capture time parameters, reference image parameters, reference time parameters, motion capture angles, motion capture positions, standby images, and captured image parameters;

[0171] A memory, used to store a program of a control method of any of the above-mentioned motion capture camera reference methods;

[0172] The program in the memory can be loaded and executed by the processor to implement any of the above-mentioned control methods of the motion capture camera reference method.

[0173] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0174] An embodiment of the present invention provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned motion capture camera reference method.

[0175] Computer storage media include, for example, various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0176] Based on the same inventive concept, an embodiment of the present invention provides an intelligent terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned motion capture camera reference method.

[0177] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0178] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A motion capture camera reference method, characterized in that: include: Controlling a preset motion capture camera to obtain motion capture feature parameters and motion capture time parameters, and controlling a preset reference camera to obtain reference image parameters and reference time parameters, wherein the reference camera refers to a device used to obtain a color high frame rate image to form a reference image parameter, and the reference camera and the motion capture camera use the ptpv1 protocol to achieve time synchronization; Matching a motion model from a preset motion model building model according to motion capture feature parameters; Judging whether there is a model abnormality from a preset model abnormality judgment model according to the motion model; When there is a model abnormality, mapping image parameters and mapping time parameters are matched from a preset image mapping model according to the motion capture feature parameters, the motion capture time parameters, the reference image parameters and the reference time parameters; identifying abnormal time parameters from a preset action abnormality recognition model according to the mapped image parameters and the mapped time parameters; According to the abnormal time parameter, the motion capture time parameter and the motion capture feature parameter, a removal feature parameter is matched from a preset abnormal removal model.

2. A motion capture camera reference method according to claim 1, characterized in that: Also included is an image display method, the image display method comprising: Select the required resolution according to the preset resolution determination method; Matching a reference frame rate from a preset frame rate matching model according to the required resolution; Matching a display image from a preset display image matching model according to a reference frame rate, a required resolution and reference image parameters; Control the preset display device to display the display image.

3. A motion capture camera reference method according to claim 2, characterized in that: The image display method further includes: Selecting a required frame rate according to a preset frame rate determination method; Match the reference resolution from the preset resolution matching model according to the required frame rate; Matching a display image from a preset display image matching model according to a reference resolution, a required frame rate, and reference image parameters; Control the preset display device to display the display image.

4. The motion capture camera reference method according to claim 1, characterized in that: Also included is a feature tracking method, the feature tracking method comprising: Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to the reference image parameters; When there is a motion feature in the picture, a reference offset is calculated from a preset reference offset calculation model based on the motion feature; When the reference offset exceeds the preset offset range, matching the rotation angle from a preset rotation angle matching model according to the reference offset; Controls the preset reference camera rotation according to the rotation angle.

5. A motion capture camera reference method according to claim 4, characterized in that: The feature tracking method further includes: When there are motion features in the picture, the feature position is matched from a preset feature position matching model based on the motion features, and the motion capture angle and motion capture position of the motion capture camera are obtained; Calculating a motion capture offset from a preset motion capture offset calculation model according to the feature position, the motion capture angle, and the motion capture position; When the motion capture offset exceeds the preset capture interval, the capture angle is matched from the preset capture angle matching model according to the feature position and the motion capture position; Calculate the difference between the capture angle and the motion capture angle and define it as the motion capture angle; Control the rotation of a preset motion capture camera according to the motion capture angle, and match the motion capture distance from a preset motion capture distance matching model according to the feature position and the motion capture position; When the motion capture distance exceeds the preset capture interval, the feature 3D is matched from the preset feature 3D recognition model based on the motion features; Matching the capture distance from a preset capture distance matching model according to the three-dimensional feature; Calculate the difference between the capture distance and the motion capture distance and define it as the moving distance; Matching the moving stroke from a preset moving stroke matching model according to the moving distance and the capture angle; Control the movement of the vehicle preset under the motion capture camera according to the movement stroke.

6. A motion capture camera reference method according to claim 4, characterized in that: Also included is a standby control method, the standby control method comprising: When there is no motion feature in the picture, the preset reference camera is controlled to obtain a standby image according to the preset standby frequency, and the preset motion capture camera is controlled to stop obtaining parameters; Determine whether there is a motion feature in the picture from a preset motion feature recognition model according to the standby image; When there are motion features in the picture, the capture frequency is matched from a preset capture frequency matching model based on the motion features; Controlling a preset reference camera to obtain captured image parameters and reference time parameters according to the capture frequency; According to the captured image parameters, reference image parameters are matched from a preset reference image optimization model, and a preset motion capture camera is controlled to obtain motion capture feature parameters and motion capture time parameters.

7. The motion capture camera reference method according to claim 1, characterized in that: It also includes a characteristic parameter optimization method, which includes: Matching the contrast image parameters from a preset contrast image matching model according to the reference image parameters, the reference time parameters and the abnormal time parameters; Extracting reference feature parameters from a preset reference feature extraction model according to the comparison image parameters; Matching optimized feature parameters from a preset feature parameter optimization model according to reference feature parameters, abnormal time parameters, motion capture feature parameters and motion capture time parameters; The merged feature parameters are matched from the preset feature merging model according to the optimized feature parameters, the abnormal time parameters and the removed feature parameters.

8. A motion capture camera reference system, characterized in that: include: An acquisition module, used to acquire motion capture feature parameters, motion capture time parameters, reference image parameters, reference time parameters, motion capture angles, motion capture positions, standby images, and captured image parameters; A memory for storing a program of a control method of a motion capture camera reference method as claimed in any one of claims 1 to 7; The program in the memory can be loaded and executed by the processor and implements a control method for a motion capture camera reference method as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and executes a motion capture camera reference method as claimed in any one of claims 1 to 7.

10. An intelligent terminal, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes a motion capture camera reference method as claimed in any one of claims 1 to 7.

Images