A big data monitoring system with face recognition function that can be added

By evaluating the time synchronization and lens distortion consistency of multi-camera systems, combining modeling quality evaluation and load analysis modules, the image processing parameters are dynamically adjusted, and the problem of insufficient performance of big data monitoring systems in the existing technology under high load is solved, and the problem of insufficient performance of big data monitoring systems under high load is realized. Efficient and stable three-dimensional face recognition and image capture.

CN118736138BActive Publication Date: 2024-11-05DALIAN HUIZHIYEXUE NETWORK TECHNOLOGY CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411208988.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-05
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

Existing big data monitoring systems are difficult to achieve an effective balance between processing complexity and modeling quality, resulting in poor time synchronization and inconsistent image correction, which affects the accuracy of three-dimensional modeling and system performance, especially in high load conditions, which are difficult to dynamically Adjusting processing parameters leads to insufficient performance and stability.

Method used

By analyzing the coordination effect between multiple cameras, calculating time synchronization accuracy and lens distortion consistency, quantifying system performance, comprehensive processing evaluation results to obtain adaptability coefficients, ensuring image fusion and three-dimensional modeling quality, and through the modeling quality evaluation module, The system load analysis module and the image capture optimization module work together to dynamically adjust image processing parameters and optimize system performance.

Benefits of technology

It realizes efficient and stable operation of multi-camera systems in high-precision applications, improves image fusion accuracy and three-dimensional modeling quality, reduces calculation pressure, meets application needs such as high-precision face recognition and virtual reality, and continuously optimizes the system and Stable operation provides technical support.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118736138B_ABST
    Figure CN118736138B_ABST
Patent Text Reader

Abstract

The present invention discloses a big data monitoring system with an addable facial recognition function, which specifically relates to the field of image monitoring and is used to solve the problem of image processing load and quality balance. The system captures the image data of the target object after analyzing and evaluating the coordination effect of multiple cameras, ensuring the acquisition of high-quality images that meet the requirements. The efficient and stable operation of the multi-camera three-dimensional face recognition system is achieved through the collaborative work of the modeling quality assessment module, the system load analysis module and the image capture optimization module. The system can dynamically adjust the number of images based on the model quality and processing load, and find the optimal balance between quality and processing load through iterative optimization, thereby reducing computing pressure, improving modeling efficiency and ensuring model quality, so that it can still maintain excellent performance under high load, meet high-precision application requirements, and provide guarantees for continuous optimization and stable operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image monitoring, and more specifically, to a big data monitoring system with an addable facial recognition function. Background Art

[0002] In big data surveillance systems, multi-camera systems with facial recognition capabilities can achieve high-precision 3D face modeling by capturing images of target objects from different angles and obtaining multi-view data. With the growing demand for facial recognition technology in security monitoring, identity authentication and other fields, the requirements for image data quality and system performance are also increasing. Multi-camera systems need to provide accurate 3D models while ensuring time synchronization and image quality consistency to meet the needs of high-precision applications.

[0003] Existing big data monitoring systems have difficulty achieving an effective balance between processing complexity and modeling quality. Poor time synchronization will lead to poor image data fusion, which in turn affects the accuracy of 3D modeling; and inconsistent image correction will make it impossible to effectively align images taken by different cameras. In addition, existing systems have difficulty dynamically adjusting processing parameters under high load conditions, and are unable to optimize resource usage while ensuring modeling quality, resulting in insufficient system performance and stability. Therefore, there is an urgent need for a solution that can comprehensively analyze and evaluate the coordination effect of multiple cameras to ensure that the system can still balance modeling quality and processing complexity under high load and meet the high-precision application requirements of big data monitoring systems.

[0004] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a big data monitoring system with an addable face recognition function. The present invention ensures that it meets the requirements for collecting target object images by analyzing and evaluating the coordination effect between multiple cameras. By calculating the time synchronization accuracy and lens distortion consistency, the performance of the multi-camera system is quantified, thereby improving the quality of image fusion and three-dimensional modeling. The evaluation results are comprehensively processed to obtain an adaptability coefficient, which is used to evaluate the overall performance of the system to ensure its adaptability and reliability in high-precision applications. Through these steps, multi-angle facial images of the target object can be effectively captured, and high-quality image data can be provided for subsequent processing. Further, through the collaborative work of the modeling quality evaluation module, the system load analysis module and the image capture optimization module, the efficient and stable operation of the multi-camera three-dimensional face recognition system is achieved. The modeling quality evaluation module evaluates the generated three-dimensional model to ensure that the quality meets the standard; the system load analysis module monitors the load and speed in real time, identifies the performance bottleneck, and triggers the optimization mechanism when the load reaches the critical point; the image capture optimization module dynamically adjusts the number of images according to the difference threshold, and finds the best balance between quality and system pressure through iterative optimization, thereby reducing the computing pressure, improving the modeling efficiency and ensuring the model quality. The system can maintain excellent performance under high load, meet the application requirements of high-precision face recognition and virtual reality, and provide technical guarantee for the continuous optimization and stable operation of the system.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A big data monitoring system with an addable facial recognition function includes: a facial image capturing module, a stereo modeling module, a modeling quality assessment module, a system load analysis module, and an image capturing optimization module;

[0008] Facial image capture module: Capture the image of the target object through a multi-angle camera, obtain multi-view data, evaluate the coordination effect, ensure data synchronization and consistency, and transmit the processed data to the modeling module;

[0009] Stereo modeling module: stitches and fuses images captured at multiple angles, uses feature points in the images to perform 3D modeling, generates 3D models, and transmits the generated 3D model data to the modeling quality assessment module and the system load analysis module;

[0010] Modeling quality assessment module: compares the original image data with the generated 3D model data. If the model quality is found to be substandard, the image processing and modeling parameters are readjusted and the assessment result data is transmitted to the system load analysis module and the image capture optimization module.

[0011] System load analysis module: Combines and analyzes the current modeling load and model quality to determine the processing load. When the processing load reaches a critical point, the optimization mechanism is triggered and the processing load analysis result data is passed to the image capture optimization module.

[0012] Image capture optimization module: In the process of capturing the facial image of the target object, a group of images whose differences meet the difference standard are selected, and the preliminary three-dimensional model generated by this is compared with the original image to verify whether the modeling quality meets the standard. If the quality meets the standard, the difference standard is dynamically adjusted, and it is iterated until the modeling quality meets the standard and the processing load does not reach the critical point. If the processing load still cannot reach the critical point after multiple iterations, an early warning reminder is issued.

[0013] In a preferred embodiment, the operation process of capturing the facial image module includes the following:

[0014] By collecting synchronization information and distortion information, it is analyzed and evaluated whether the coordination effect between multiple cameras meets the qualifications for collecting images of the target object, where the synchronization information includes the synchronization delay jitter index, and the distortion information includes the lens distortion consistency index.

[0015] In a preferred embodiment, the process of obtaining the synchronization delay jitter index is as follows:

[0016] a1, calibrate the timestamp of each camera through the synchronization signal to ensure that the timestamps of all cameras are synchronized;

[0017] Place synchronization markers in multiple camera fields of view and capture synchronized events at high frame rates;

[0018] Record the timestamp sequence of each camera capturing the changes of the synchronization marker points;

[0019] a2, for each pair of cameras, extract the timestamp sequence of the synchronous events captured by them;

[0020] Calculate the time delay difference of each pair of synchronous events;

[0021] The delay differences are grouped and counted, and a frequency distribution of the delay differences is formed through a cross-correlation function;

[0022] a3, find the location of the maximum value in the cross-correlation function;

[0023] a4, for all synchronization events, calculate their time delay difference, and count their standard deviations and mark them as synchronization delay jitter parameters;

[0024] a5. Perform steps a1 to a4 for each pair of cameras to obtain synchronization delay jitter parameters of each pair of cameras, and take the maximum value of all synchronization delay jitter parameters and mark it as the synchronization delay jitter index.

[0025] In a preferred embodiment, the process of obtaining the lens distortion consistency index is as follows:

[0026] b1, ensure the time synchronization of all cameras, and take a series of images at the same time to record the image sequence at multiple time points;

[0027] b2, preprocess the image sequence captured by each camera;

[0028] Use the classic optical flow algorithm to calculate the optical flow field between adjacent images. For each pair of adjacent images, calculate the optical flow vector of the pixel points to form an optical flow field.

[0029] b3, for each pair of cameras, compare the optical flow fields at the same time point and calculate the difference in optical flow vectors, which is measured by the difference in the size and direction of the vectors;

[0030] b4, count the differences in optical flow vectors at all time points, and select the maximum value of the optical flow vector difference as the lens distortion consistency index.

[0031] In a preferred embodiment, the synchronization delay jitter index and the lens distortion consistency index are comprehensively processed to obtain the camera modeling adaptability coefficient; the camera modeling adaptability coefficient is compared with the applicable threshold: when the camera modeling adaptability coefficient is greater than or equal to the applicable threshold, a green signal is generated; conversely, when the camera modeling adaptability coefficient is less than the applicable threshold, a red signal is generated, indicating that the camera parameters need to be adjusted.

[0032] In a preferred embodiment, the operation process of the modeling quality assessment module includes the following:

[0033] Collect and organize the raw image data captured by all cameras;

[0034] Receive and organize the three-dimensional model data generated by the three-dimensional modeling module;

[0035] Extracting a 3D point cloud from the 3D model data and defining a voxel grid of a standard size in 3D space;

[0036] Through the voxelization method, the three-dimensional space is divided into a series of small voxels, and the number of points in each voxel is counted to obtain the density distribution of the point cloud;

[0037] Convert the original image data into a coordinate system consistent with the 3D point cloud and calculate the density distribution under the same voxel grid;

[0038] The face is divided into several key regions, and in each key feature region, the point cloud density of the original image data and the 3D model is calculated respectively;

[0039] Compare the density differences of each key feature area. The specific evaluation criteria are as follows: for each key feature area, calculate the absolute value and sum of the density differences. If the sum of the density differences of any key feature area exceeds the preset threshold, the model quality is considered to be substandard.

[0040] Based on the density difference results of the local structure analysis, determine whether the model quality meets the standard: the sum of the density differences of all key feature areas must be within the preset threshold; if the sum of the density differences of all key feature areas is within the preset threshold, the model quality meets the standard; otherwise, the model quality does not meet the standard.

[0041] In a preferred embodiment, the operation process of the system load analysis module includes the following contents:

[0042] Collect relevant data of the operation of the three-dimensional modeling module and the modeling quality assessment module in real time, including the start and end time of each modeling, the model quality assessment results, and the resources used in the modeling process;

[0043] Calculate the number of face modeling completed per unit time. The specific method is to count the number of modelings successfully completed within a fixed time window to obtain the face modeling speed per unit time.

[0044] Based on the results of the modeling quality assessment module, statistics are collected to determine whether the model quality meets the standards each time;

[0045] In a fixed time window, the proportion of the number of modelings that meet the quality standards to the total number of modelings is calculated to obtain the quality standard ratio;

[0046] Based on the face modeling speed and quality compliance ratio per unit time, the harmonic mean method is used to comprehensively evaluate the system processing pressure and obtain the modeling task saturation;

[0047] Compare the modeling task saturation and the saturation threshold: when the modeling task saturation is greater than or equal to the saturation threshold, it indicates that the system's processing load and modeling quality per unit time are within an acceptable range; when the modeling task saturation is less than the saturation threshold, it indicates that the system's processing load is high or the proportion of modeling quality that meets the standard is low, and the processing load has reached a critical situation.

[0048] In a preferred embodiment, the operation process of the image capture optimization module includes the following:

[0049] d1, when the saturation of the modeling task is less than the saturation threshold, the captured images are preliminarily screened to select a group of images whose differences reach the difference threshold;

[0050] d2, generating a preliminary three-dimensional model for the filtered image group through the stereo modeling module;

[0051] d3, compares the preliminary 3D model with the original image data through the modeling quality assessment module;

[0052] d4, if the model quality of the preliminary three-dimensional model is determined to be up to standard in step d3, the difference threshold is dynamically adjusted;

[0053] The specific method of dynamically adjusting the difference threshold adopts an incremental strategy, that is, multiplying the current difference threshold by a fixed incremental factor in each iteration;

[0054] d5, regenerate the 3D model using the adjusted image group, evaluate the quality of the 3D model again according to step d3, and repeat steps d1 to d4 until the model quality meets the standard and the saturation of the modeling task is greater than or equal to the saturation threshold;

[0055] d6: If, under the premise of meeting the model quality standard, steps d1 to d4 are repeated for the rated number of times, if the condition that the saturation of the modeling task is greater than or equal to the saturation threshold is still not met, an early warning reminder is issued.

[0056] The technical effects and advantages of the big data monitoring system with an addable facial recognition function provided by the present invention are as follows:

[0057] 1. The present invention ensures that the requirements for capturing images of target objects are met by analyzing and evaluating the coordination effect between multiple cameras. By calculating the synchronization delay jitter index and the lens distortion consistency index, the time synchronization accuracy and lens distortion consistency level of the multi-camera system are quantified, thereby improving the accuracy of image fusion and the quality of three-dimensional modeling. The camera modeling adaptability coefficient is obtained by comprehensive processing of these two indexes, which is used to evaluate the overall performance of the system and ensure its adaptability and reliability in high-precision applications. Through these steps, multi-angle facial images of target objects can be effectively captured, providing high-quality image data for subsequent processing.

[0058] 2. The present invention realizes the efficient and stable operation of the multi-camera three-dimensional face recognition system through the collaborative work of the modeling quality assessment module, the system load analysis module and the image capture optimization module. The modeling quality assessment module accurately evaluates the generated three-dimensional model to ensure that the modeling quality meets the standard; the system load analysis module monitors the system load and modeling speed in real time, uses a comprehensive evaluation method to identify the system performance bottleneck, and triggers the optimization mechanism when the load reaches the critical point; the image capture optimization module dynamically adjusts the number of images involved in the modeling according to the difference threshold, and finds the optimal balance between modeling quality and system pressure through iterative optimization, thereby reducing computing pressure, improving modeling efficiency and ensuring model quality. The system can still maintain excellent performance under high load, meet the application requirements of high-precision face recognition and virtual reality, and provide technical guarantee for the continuous optimization and stable operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of a big data monitoring system with an addable facial recognition function according to the present invention;

[0060] Figure 2 The present invention is a schematic diagram of the steps of the operation process of an image capture optimization module of a big data monitoring system with an addable facial recognition function. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Embodiment 1, Figure 1 The present invention provides a big data monitoring system with an addable face recognition function, comprising: a facial image capturing module, a stereo modeling module, a modeling quality assessment module, a system load analysis module and an image capturing optimization module;

[0063] Facial image capture module: Capture the image of the target object through a multi-angle camera, obtain multi-view data, evaluate the coordination effect, ensure data synchronization and consistency, and transmit the processed data to the modeling module;

[0064] Stereo modeling module: stitches and fuses images captured at multiple angles, uses feature points in the images to perform 3D modeling, generates 3D models, and transmits the generated 3D model data to the modeling quality assessment module and the system load analysis module;

[0065] Modeling quality assessment module: compares the original image data with the generated 3D model data. If the model quality is found to be substandard, the image processing and modeling parameters are readjusted and the assessment result data is transmitted to the system load analysis module and the image capture optimization module.

[0066] System load analysis module: Combines and analyzes the current modeling load and model quality to determine the processing load. When the processing load reaches a critical point, the optimization mechanism is triggered and the processing load analysis result data is passed to the image capture optimization module.

[0067] Image capture optimization module: In the process of capturing the facial image of the target object, a group of images whose differences meet the difference standard are selected, and the preliminary three-dimensional model generated by this is compared with the original image to verify whether the modeling quality meets the standard. If the quality meets the standard, the difference standard is dynamically adjusted, and it is iterated until the modeling quality meets the standard and the processing load does not reach the critical point. If the processing load still cannot reach the critical point after multiple iterations, an early warning reminder is issued.

[0068] It should be noted that an addable monitoring system means that the system has the ability to be flexibly expanded, and can add or upgrade facial recognition functions or other related functions through modular design according to actual application needs. Specifically, the design of the system allows users to achieve functions such as enhanced data processing capabilities, improved recognition accuracy, adding more cameras, or introducing new recognition algorithms by adding new modules or upgrading existing modules after initial deployment according to changing needs. This modular design ensures the adaptability and scalability of the system in different application scenarios, allowing users to flexibly adjust system functions according to actual needs to cope with ever-changing monitoring and recognition tasks.

[0069] The operation process of the facial image capture module includes the following:

[0070] In order to ensure that the coordination effect between multiple cameras meets the qualifications for capturing images of the target object, it is first necessary to analyze and evaluate the coordination effect between the multiple cameras. By collecting synchronization information and distortion information, it is analyzed and evaluated whether the coordination effect between the multiple cameras meets the qualifications for capturing images of the target object, where the synchronization information includes the synchronization delay jitter index, and the distortion information includes the lens distortion consistency index.

[0071] In a multi-camera system, time synchronization is a key factor to ensure that all cameras capture the target object at the same time. Time delay variation, that is, the difference in timestamps when each camera records the same event, will directly affect the quality of image fusion. The images captured by multiple cameras need to be highly consistent in time to ensure the accuracy and consistency of the model in the subsequent image processing and 3D modeling process. If the time delay is large, there may be a significant time difference between the images taken by different cameras, resulting in inconsistent position and posture of the target object, which in turn affects image registration and the construction of 3D models, resulting in distortion and errors. Therefore, it is of great significance to calculate the synchronization delay jitter index and quantify the time synchronization accuracy of the multi-camera system. A smaller synchronization delay jitter index means that the time delay variation is small and the camera synchronization is high, which can effectively improve the accuracy of image fusion and the quality of 3D modeling. The calculation of this index provides a scientific basis for evaluating and optimizing the performance of multi-camera systems, ensuring that better modeling results can be obtained in high-precision applications such as face recognition, virtual reality, and high-precision measurement.

[0072] The process of obtaining the synchronization delay jitter index is as follows:

[0073] a1, calibrate the timestamp of each camera through a synchronization signal (such as NTP or GPS) to ensure that the timestamps of all cameras are synchronized;

[0074] Place synchronized markers (such as LED lights) in the field of view of multiple cameras and capture synchronized events (such as LED flashes) at a high frame rate.

[0075] Record the timestamp sequence of each camera capturing the changes of the synchronization marker points. These timestamps will serve as the basic data for subsequent analysis.

[0076] a2. For each pair of cameras, extract the timestamp sequence of the synchronous events captured by them. For example, suppose the timestamp sequences of the two cameras are and ,in and Indicates The timestamp of the synchronization event on camera A and camera B.

[0077] Calculate the time delay difference for each pair of synchronous events ;

[0078] The delay differences are grouped and counted, and the frequency distribution of the delay differences is formed through the cross-correlation function. Reflects different delay values The frequency of occurrence.

[0079] a3, in the cross-correlation function Find the maximum value in , which corresponds to the most common time delay between cameras.

[0080] a4, for all synchronization events, calculate their time delay differences, and count their standard deviations and mark them as synchronization delay jitter parameters. Use statistical methods to calculate the standard deviation of the delay difference. The larger the standard deviation, the more serious the synchronization delay jitter.

[0081] a5. Perform steps a1 to a4 for each pair of cameras to obtain synchronization delay jitter parameters of each pair of cameras, and take the maximum value of all synchronization delay jitter parameters and mark it as the synchronization delay jitter index.

[0082] The synchronization delay jitter index is used to represent and reflect the time synchronization accuracy between different cameras in a multi-camera system. Specifically, the larger the jitter index, the greater the time delay variation between cameras, that is, the lower the synchronization accuracy; the smaller the jitter index, the smaller the time delay variation between cameras, that is, the higher the synchronization accuracy. The synchronization delay jitter index is crucial to the quality of generating 3D face modeling. If the jitter index is large, it means that the images captured by the camera are inconsistent in time, which may cause errors in the image fusion and 3D modeling process, affecting the accuracy and quality of modeling; conversely, if the jitter index is small, the image captured by the camera has a high consistency in time, which helps to improve the accuracy of image fusion, thereby generating a higher quality 3D face model. Therefore, reducing the synchronization delay jitter index helps to improve the performance and effect of the multi-camera system in the 3D modeling process.

[0083] Among them, the process of obtaining the lens distortion consistency index is as follows:

[0084] In a multi-camera system, the consistency of lens distortion of each camera is crucial to the accuracy of image fusion and 3D modeling. Lens distortion refers to the geometric distortion of the image caused by the non-ideal characteristics of the lens. The degree and type of lens distortion of different cameras may vary, and these differences will directly affect the registration and fusion of the images collected by the multi-camera system. Inconsistent lens distortion will lead to inconsistent results of the images taken by each camera after geometric correction, which will in turn affect the superposition and matching of the images, resulting in errors and distortions in the 3D modeling process. By calculating the lens distortion consistency index, the consistency level of lens distortion of each camera in the multi-camera system can be quantified. A smaller distortion consistency index indicates that the lens distortion between cameras is more consistent, which helps to improve the accuracy of image fusion and thus generate a more accurate and high-quality 3D model; on the contrary, a larger distortion consistency index indicates that the lens distortion is inconsistent, which may lead to increased modeling errors and reduce the accuracy and authenticity of the model. Therefore, evaluating and optimizing the consistency of lens distortion is of great significance to ensure the performance of multi-camera systems in high-precision applications.

[0085] b1, ensure the time synchronization of all cameras, and take a series of images at the same time to record the image sequence at multiple time points.

[0086] b2, preprocess the image sequences captured by each camera, including denoising, contrast enhancement, etc., to improve the accuracy of optical flow calculation;

[0087] Use classic optical flow algorithms (such as the Lucas-Kanade method or the Horn-Schunck method) to calculate the optical flow field between adjacent images. For each pair of adjacent images, calculate the optical flow vector of the pixel point to form an optical flow field.

[0088] b3, for each pair of cameras, compare the optical flow fields at the same time point and calculate the difference in optical flow vectors. The difference in optical flow vectors can be measured by the difference in the magnitude and direction of the vector;

[0089] b4, count the differences in optical flow vectors at all time points, and select the maximum value of the optical flow vector difference as the lens distortion consistency index.

[0090] The lens distortion consistency index is used to represent and reflect the consistency of lens distortion of each camera in a multi-camera system. Specifically, the larger the distortion consistency index, the greater the difference in lens distortion between different cameras, that is, the worse the consistency; the smaller the distortion consistency index, the smaller the difference in lens distortion between different cameras, that is, the higher the consistency. The lens distortion consistency index is crucial to the quality of generating 3D face modeling. If the distortion consistency index is large, it means that there are significant differences in the results of distortion correction of images taken by different cameras, resulting in errors in image fusion, affecting the accuracy and authenticity of the 3D model; conversely, a smaller distortion consistency index indicates that the lens distortion consistency between cameras is high, and the images are more consistent after distortion correction, which can effectively improve the accuracy of image fusion, thereby generating a more accurate and high-quality 3D face model. Therefore, reducing the lens distortion consistency index helps to improve the performance and effect of the multi-camera system in the 3D modeling process.

[0091] The synchronization delay jitter index and the lens distortion consistency index are processed comprehensively to obtain the camera modeling adaptability coefficient, which can be calculated by the following formula, for example: ;

[0092] in represents the adaptability coefficient of camera modeling; Indicates the synchronization delay jitter index, reflecting the time synchronization accuracy between cameras; Represents the lens distortion consistency index, which reflects the consistency of lens distortion between cameras; and are weight parameters, reflecting the relative importance of synchronization delay jitter and lens distortion consistency to the overall system performance; and It is a small positive number (epsilon) that prevents the denominator from being zero, ensuring that no division by zero error occurs during calculations.

[0093] The formula takes the root of the sum of the squares of the synchronization delay jitter index and the lens distortion consistency index, which can smooth out extreme values ​​and ensure that the extreme value of a single index does not excessively affect the overall evaluation.

[0094] Weight Parameters and Used to adjust the degree of influence of synchronization delay and lens distortion on the overall system performance. These two parameters can be adjusted according to specific system and application requirements.

[0095] The larger the camera modeling adaptability coefficient value, the better the system's time synchronization and lens distortion correction effect, which is suitable for high-precision 3D face modeling; the smaller the camera modeling adaptability coefficient value, the greater the system's time synchronization and lens distortion correction problem, which is not conducive to high-precision 3D modeling. By calculating the camera modeling adaptability coefficient, the hardware performance of the multi-camera system can be systematically evaluated to ensure its adaptability and reliability in high-precision applications.

[0096] Compare the camera modeling adaptability coefficient with the applicable threshold to evaluate whether the camera system is suitable for high-precision 3D face modeling. When the camera modeling adaptability coefficient is greater than or equal to the applicable threshold, it means that the camera system has good time synchronization and lens distortion correction effects, and is suitable for high-precision 3D face modeling. At this time, a green signal is generated, indicating that the system can be used for high-precision modeling tasks. Conversely, when the camera modeling adaptability coefficient is less than the applicable threshold, it means that there are major problems with the camera system's time synchronization and lens distortion correction, which is not conducive to high-precision 3D modeling. At this time, a red signal is generated, indicating that the camera parameters need to be adjusted or the system optimization needs to be performed to improve performance.

[0097] After obtaining the green signal, multiple cameras are used to simultaneously capture the facial image of the target object. Each camera takes facial images of the target object from different angles to obtain multiple sets of image data. These image data include facial details from multiple perspectives to ensure that the complete facial information of the target object is covered.

[0098] The present invention ensures that the requirements for collecting target object images are met by analyzing and evaluating the coordination effect between multiple cameras. By calculating the synchronization delay jitter index and the lens distortion consistency index, the time synchronization accuracy and lens distortion consistency level of the multi-camera system are quantified, thereby improving the accuracy of image fusion and the quality of three-dimensional modeling. The two indexes are processed comprehensively to obtain the camera modeling adaptability coefficient, which is used to evaluate the overall performance of the system and ensure its adaptability and reliability in high-precision applications. Through these steps, multi-angle facial images of the target object can be effectively captured, providing high-quality image data for subsequent processing.

[0099] The operation process of the three-dimensional modeling module includes the following:

[0100] In the multi-camera 3D face modeling system, the stereo modeling module is one of the core components, responsible for accurately stitching and fusing facial images from different perspectives to generate high-quality 3D face models. This module uses advanced image processing technology and feature point matching algorithms to ensure seamless fusion of multi-view images, and reconstructs detailed 3D structures through triangulation and point cloud processing methods. The goal of the stereo modeling module is to maximize the accuracy and detail restoration of the 3D model, laying a solid foundation for subsequent modeling quality evaluation and system optimization. The high-precision 3D model it generates has broad application prospects in high-precision applications such as virtual reality, face recognition, and biometric recognition. Through high-quality modeling results, the reliability and effectiveness of the system in practical applications are ensured.

[0101] After obtaining the green signal, use feature detection algorithms (such as SIFT, SURF or ORB) to extract key feature points in each image; generate feature descriptors for each feature point to facilitate subsequent feature matching.

[0102] Perform feature matching between adjacent images to find corresponding feature point pairs. Based on the matched feature point pairs, estimate the geometric transformation matrix between the images and perform image stitching to construct the overall perspective image.

[0103] The spatial coordinates of the feature points and the camera parameters are used to calculate the three-dimensional coordinates of the feature points through triangulation. The three-dimensional coordinates of all feature points are collected into a three-dimensional point cloud, and the point cloud is converted into a three-dimensional model through a point cloud processing algorithm.

[0104] The generated point cloud is subjected to noise filtering to remove noise points in the point cloud to optimize the point cloud quality. Then, the texture information of the original image is mapped to the surface of the 3D model to enhance the visual effect of the model and make it more realistic and detailed.

[0105] Finally, the generated 3D model is converted into the required file format, and the generated 3D model data is passed to the modeling quality assessment module for further quality assessment and verification.

[0106] The stereo modeling module can accurately stitch and fuse facial images from multiple angles to generate a high-quality three-dimensional face model and ensure the accuracy and consistency of the model data, thus meeting the needs of high-precision three-dimensional modeling.

[0107] The operation process of the modeling quality assessment module includes the following:

[0108] In the multi-camera 3D face modeling system, generating high-quality 3D models places extremely high demands on the image fusion and modeling process. The modeling quality assessment module is designed to ensure the accuracy and consistency of the model by closely comparing and analyzing the original image data with the generated 3D model data. This module uses the density difference analysis method to carefully evaluate the density distribution of the 3D model on the local structure, which can accurately capture the geometric details and errors in key areas, ensuring the reliability and authenticity of the generated 3D face model in high-precision applications. By continuously adjusting image processing and modeling parameters and optimizing the modeling process, the modeling quality assessment module provides a scientific basis and technical support for improving the overall performance of the system. While ensuring that the model quality meets the standards, this module transmits the evaluation result data to the system load analysis module to form a closed-loop feedback mechanism to promote the continuous optimization and performance improvement of the system, thereby meeting the strict requirements of face recognition, virtual reality, and high-precision measurement.

[0109] Collect and organize the raw image data captured by all cameras to ensure that these data have the same timestamp and perspective information as the generated 3D model data.

[0110] Receive and organize the 3D model data generated by the stereo modeling module, ensure that the model data format is correct and contains all necessary 3D coordinates and texture information.

[0111] Extract 3D point clouds from 3D model data and define a standard-sized voxel grid in 3D space. Through the voxelization method, the 3D space is divided into a series of small voxels, and the number of points in each voxel is counted to obtain the density distribution of the point cloud. Then, the original image data is converted into a coordinate system consistent with the 3D point cloud, and the density distribution under the same voxel grid is calculated.

[0112] The face is divided into several key areas, including eyes, nose, and mouth. In each key feature area, the point cloud density of the original image data and the 3D model is calculated respectively. Then, the density difference of each key feature area is compared. The specific evaluation criteria are: for each key feature area, the absolute value and sum of the density difference are calculated. If the sum of the density difference of any key feature area exceeds the preset threshold, the model quality is considered to be substandard.

[0113] Based on the density difference results of the local structure analysis, determine whether the model quality meets the standard: the sum of the density differences of all key feature areas must be within the preset threshold. If the sum of the density differences of all key feature areas is within the preset threshold, the model quality meets the standard; otherwise, the model quality does not meet the standard.

[0114] If the model quality does not meet the standard, determine the image processing and modeling parameters that need to be adjusted based on the results of the local structure analysis. These adjustments may involve image preprocessing methods, feature point extraction algorithms, and modeling algorithms. After adjusting the parameters, re-perform the image processing and 3D modeling process to generate new 3D model data. Perform quality assessment on the new 3D model data again until the model quality meets the standard.

[0115] The operation process of the system load analysis module includes the following:

[0116] In the multi-camera 3D face recognition system, the system load analysis module is a key component to ensure the efficient and stable operation of the system. This module collects and analyzes the relevant data of face modeling in real time, evaluates the system's processing load and modeling quality, and ensures that the system can maintain excellent performance under high load. By calculating the face modeling speed and quality compliance ratio per unit time, the system load analysis module can keenly capture the weak links in the system performance and comprehensively evaluate the modeling task saturation through the harmonic mean method. The calculation results of the modeling task saturation reflect the current processing capacity and quality level of the system, and can effectively guide system optimization and resource allocation. When the system pressure reaches the critical value, the module will trigger the optimization mechanism, dynamically adjust the system parameters, and improve the overall performance. The system load analysis module has broad application prospects in high-precision applications such as face recognition, virtual reality, and biometric recognition, and provides technical guarantees for continuous optimization and stable operation of the system.

[0117] Collect relevant data of the operation of the three-dimensional modeling module and the modeling quality assessment module in real time, including the start and end time of each modeling, the model quality assessment results, and the resources used in the modeling process;

[0118] Calculate the number of face modeling completed per unit time. The specific method is to count the number of successful modeling completed within a fixed time window (such as 1 hour) to obtain the face modeling speed per unit time.

[0119] The rolling time window method is used to continuously update the modeling speed to ensure that the processing speed of the system can be reflected in real time.

[0120] Based on the results of the modeling quality assessment module, statistics are collected to determine whether the quality of each modeling is up to standard.

[0121] In a fixed time window, the ratio of the number of modelings that meet the quality standards to the total number of modelings is calculated to obtain the quality standard ratio.

[0122] Based on the face modeling speed and quality compliance ratio per unit time, the harmonic mean method is used to comprehensively evaluate the system processing pressure and obtain the modeling task saturation. For example, it is calculated by the following formula: ;

[0123] Where P represents the saturation of the modeling task; S represents the speed of face modeling per unit time; Q represents the quality compliance ratio; and are the weight coefficients of modeling speed and quality compliance ratio respectively.

[0124] The harmonic mean method is more sensitive to extreme values ​​and can effectively reflect the weaker link in system performance. In the system, if a certain parameter is significantly low, the harmonic mean method will significantly reduce the overall index, which can promptly remind you that optimization and adjustment are needed.

[0125] The modeling task saturation is used to reflect and represent the modeling processing load and quality compliance of the multi-camera face recognition system within a certain period of time. Specifically, the greater the modeling task saturation, the more face modeling tasks the system can efficiently process per unit time, and the higher the quality compliance ratio is, which means that the system's processing power and modeling accuracy are at a high level; on the contrary, the smaller the modeling task saturation, the slower the face modeling speed of the system per unit time, or the lower the modeling quality compliance ratio is, reflecting that the system has a large load pressure or quality problems, suggesting that optimization and adjustment are needed to improve the overall performance and modeling effect of the system. By comprehensively evaluating the modeling speed and quality compliance ratio, the modeling task saturation can keenly capture the weak links in the system performance, ensure that corresponding measures are taken in a timely manner, and maintain the system running in an efficient and high-quality state.

[0126] Compare the modeling task saturation to the saturation threshold:

[0127] When the modeling task saturation is greater than or equal to the saturation threshold, it indicates that the system's processing load and modeling quality per unit time are within an acceptable range, the system is in good operating condition, can efficiently handle the current task volume, and the modeling quality reaches a high standard. This means that the system's currently configured resources and parameter settings are reasonable and can meet the needs of high-precision face modeling. In this case, the system can continue to operate according to the existing configuration, and regular monitoring can be performed to ensure continued stable performance.

[0128] When the modeling task saturation is less than the saturation threshold, it indicates that the system's processing load is high or the modeling quality compliance ratio is low. The system may be at risk of overload or have modeling quality issues, and the processing load has reached a critical situation. This may cause the system to be unable to complete the modeling task in a timely and effective manner, affecting overall performance and user experience. In order to cope with this situation, it is necessary to trigger the optimization mechanism, dynamically adjust system parameters, increase computing resources, or improve image processing and modeling algorithms to reduce system load, increase modeling speed and quality compliance ratio, and thus restore the system's efficient operation state.

[0129] like Figure 2 As shown, the operation process of the image capture optimization module includes the following:

[0130] d1, when the saturation of the modeling task is less than the saturation threshold, the captured images are preliminarily screened and a group of images whose differences reach the difference threshold are selected, where the differences are based on changes in image content, including but not limited to angles, lighting, and facial expressions. The differences between images are calculated using the structural similarity index (SSIM), and the similarity of each pair of images is measured by comparing their SSIM values. Image pairs with lower SSIM values ​​are selected to indicate that the difference between the images is larger.

[0131] d2, generate a preliminary three-dimensional model for the filtered image group through the stereo modeling module.

[0132] d3, the preliminary 3D model is compared with the original image data through the modeling quality assessment module.

[0133] d4. If the model quality of the preliminary 3D model is judged to be up to standard in step d3, the difference threshold is dynamically adjusted to reduce the number of images involved in 3D modeling. The specific steps are to gradually increase the difference threshold standard, select image groups with larger differences to participate in modeling, and reduce the number of images involved in 3D modeling, thereby reducing the system computing pressure.

[0134] The specific method of dynamically adjusting the difference threshold can adopt an incremental strategy, that is, in each iteration, the current difference threshold is multiplied by a fixed incremental factor, such as 1.1, to gradually increase the threshold standard of image difference, and select image groups with larger differences to participate in modeling, thereby reducing the number of images involved in three-dimensional modeling and reducing the system calculation pressure.

[0135] d5, regenerate the 3D model using the adjusted image group, and re-evaluate the quality of the 3D model according to step d3, repeat steps d1 to d4, continuously increase the threshold of image difference, generate the 3D model and verify the quality, gradually reduce the number of images, reduce the system computing pressure, and ensure that the model quality is stable and up to standard. Through multiple iterations, until the model quality meets the standard and the saturation of the modeling task is greater than or equal to the saturation threshold; thus, the best balance between modeling quality and system pressure is found, that is, the number of images involved in modeling is minimized while ensuring that the model quality meets the standard.

[0136] d6: If, under the premise of meeting the model quality standard, steps d1 to d4 are repeated for the rated number of times, if the condition that the saturation of the modeling task is greater than or equal to the saturation threshold is still not met, an early warning reminder is issued, indicating that the processing load cannot be optimized and human intervention is required.

[0137] The present invention realizes the efficient and stable operation of the multi-camera three-dimensional face recognition system through the collaborative work of the modeling quality assessment module, the system load analysis module and the image capture optimization module. The modeling quality assessment module accurately evaluates the generated three-dimensional model to ensure that the modeling quality meets the standard; the system load analysis module monitors the system load and modeling speed in real time, uses a comprehensive evaluation method to identify the system performance bottleneck, and triggers the optimization mechanism when the load reaches the critical point; the image capture optimization module dynamically adjusts the number of images involved in the modeling according to the difference threshold, and finds the optimal balance point between the modeling quality and the system pressure through iterative optimization, thereby reducing the computing pressure, improving the modeling efficiency and ensuring the model quality. This enables the system to maintain excellent performance under high load, meet the application requirements of high-precision face recognition and virtual reality, and provide technical guarantee for the continuous optimization and stable operation of the system.

[0138] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0139] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0140] It should be noted that, in this article, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A big data monitoring system with an add-on facial recognition function, characterized in that: include: facial image capture module, stereo modeling module, modeling quality assessment module, system load analysis module and image capture optimization module; Facial image capture module: Capture the image of the target object through a multi-angle camera, obtain multi-view data, evaluate the coordination effect, ensure data synchronization and consistency, and transmit the processed data to the modeling module; Stereo modeling module: stitches and fuses images captured at multiple angles, uses feature points in the images to perform 3D modeling, generates 3D models, and transmits the generated 3D model data to the modeling quality assessment module and the system load analysis module; Modeling quality assessment module: compares the original image data with the generated 3D model data. If the model quality is found to be substandard, the image processing and modeling parameters are readjusted and the assessment result data is transmitted to the system load analysis module and the image capture optimization module. System load analysis module: Combines and analyzes the current modeling load and model quality to determine the processing load. When the processing load reaches a critical point, the optimization mechanism is triggered and the processing load analysis result data is passed to the image capture optimization module. Image capture optimization module: In the process of capturing the facial image of the target object, a group of images whose differences meet the difference standard are selected, and the preliminary 3D model generated by them is compared with the original image to verify whether the modeling quality meets the standard. If the quality meets the standard, the difference standard is dynamically adjusted, and it is iterated until the modeling quality meets the standard and the processing load does not reach the critical value. If the processing load still fails to reach the critical value after multiple iterations, an early warning reminder is issued; The operation process of the facial image capture module includes the following: By collecting synchronization information and distortion information, the coordination effect between multiple cameras is analyzed and evaluated to see whether it satisfies the qualification of collecting images of target objects, wherein the synchronization information includes the synchronization delay jitter index, and the distortion information includes the lens distortion consistency index; The process of obtaining the synchronization delay jitter index is as follows: a1, calibrate the timestamp of each camera through the synchronization signal to ensure that the timestamps of all cameras are synchronized; Place synchronization markers in multiple camera fields of view and capture synchronized events at high frame rates; Record the timestamp sequence of each camera capturing the changes of the synchronization marker points; a2, for each pair of cameras, extract the timestamp sequence of the synchronous events captured by them; Calculate the time delay difference of each pair of synchronous events; The delay differences are grouped and counted, and a frequency distribution of the delay differences is formed through a cross-correlation function; a3, find the location of the maximum value in the cross-correlation function; a4, for all synchronization events, calculate their time delay difference, and count their standard deviations and mark them as synchronization delay jitter parameters; a5, performing step a1 to step a4 analysis on each pair of cameras to obtain synchronization delay jitter parameters of each pair of cameras, taking the maximum value of all synchronization delay jitter parameters and marking it as the synchronization delay jitter index; Among them, the process of obtaining the lens distortion consistency index is as follows: b1, ensure the time synchronization of all cameras, and take a series of images at the same time to record the image sequence at multiple time points; b2, preprocess the image sequence captured by each camera; Use the classic optical flow algorithm to calculate the optical flow field between adjacent images. For each pair of adjacent images, calculate the optical flow vector of the pixel points to form an optical flow field. b3, for each pair of cameras, compare the optical flow fields at the same time point and calculate the difference in optical flow vectors, which is measured by the difference in the size and direction of the vectors; b4, count the differences in optical flow vectors at all time points, and select the maximum value of the optical flow vector difference as the lens distortion consistency index; The operation process of capturing facial images module also includes the following: The synchronization delay jitter index and the lens distortion consistency index are comprehensively processed to obtain the camera modeling adaptability coefficient; the camera modeling adaptability coefficient is compared with the applicable threshold: when the camera modeling adaptability coefficient is greater than or equal to the applicable threshold, a green signal is generated; conversely, when the camera modeling adaptability coefficient is less than the applicable threshold, a red signal is generated, indicating that the camera parameters need to be adjusted.

2. A big data monitoring system with an addable facial recognition function according to claim 1, characterized in that: The operation process of the modeling quality assessment module includes the following: Collect and organize the raw image data captured by all cameras; Receive and organize the three-dimensional model data generated by the three-dimensional modeling module; Extracting a 3D point cloud from the 3D model data and defining a voxel grid of a standard size in 3D space; Through the voxelization method, the three-dimensional space is divided into a series of small voxels, and the number of points in each voxel is counted to obtain the density distribution of the point cloud; Convert the original image data into a coordinate system consistent with the 3D point cloud and calculate the density distribution under the same voxel grid; The face is divided into several key regions, and in each key feature region, the point cloud density of the original image data and the 3D model is calculated respectively; Compare the density differences of each key feature area. The specific evaluation criteria are as follows: for each key feature area, calculate the absolute value and sum of the density differences. If the sum of the density differences of any key feature area exceeds the preset threshold, the model quality is considered to be substandard. Based on the density difference results of the local structure analysis, determine whether the model quality meets the standard: the sum of the density differences of all key feature areas must be within the preset threshold; if the sum of the density differences of all key feature areas is within the preset threshold, the model quality meets the standard; otherwise, the model quality does not meet the standard.

3. A big data monitoring system with an addable facial recognition function according to claim 2, characterized in that: The operation process of the system load analysis module includes the following: Collect relevant data of the operation of the three-dimensional modeling module and the modeling quality assessment module in real time, including the start and end time of each modeling, the model quality assessment results, and the resources used in the modeling process; Calculate the number of face modeling completed per unit time. The specific method is to count the number of modelings successfully completed within a fixed time window to obtain the face modeling speed per unit time. Based on the results of the modeling quality assessment module, statistics are collected to determine whether the model quality meets the standards each time; In a fixed time window, the proportion of the number of modelings that meet the quality standards to the total number of modelings is calculated to obtain the quality standard ratio; Based on the face modeling speed and quality compliance ratio per unit time, the harmonic mean method is used to comprehensively evaluate the system processing pressure and obtain the modeling task saturation; Compare the modeling task saturation and the saturation threshold: when the modeling task saturation is greater than or equal to the saturation threshold, it indicates that the system's processing load and modeling quality per unit time are within an acceptable range; when the modeling task saturation is less than the saturation threshold, it indicates that the system's processing load is high or the proportion of modeling quality that meets the standard is low, and the processing load has reached a critical situation.

4. The big data monitoring system with an addable facial recognition function according to claim 3, characterized in that: The operation process of the image capture optimization module includes the following: d1, when the saturation of the modeling task is less than the saturation threshold, the captured images are preliminarily screened to select a group of images whose differences reach the difference threshold; d2, generating a preliminary three-dimensional model for the filtered image group through the stereo modeling module; d3, compares the preliminary 3D model with the original image data through the modeling quality assessment module; d4, if the model quality of the preliminary three-dimensional model is determined to be up to standard in step d3, the difference threshold is dynamically adjusted; The specific method of dynamically adjusting the difference threshold adopts an incremental strategy, that is, multiplying the current difference threshold by a fixed incremental factor in each iteration; d5, regenerate the 3D model using the adjusted image group, evaluate the quality of the 3D model again according to step d3, and repeat steps d1 to d4 until the model quality meets the standard and the saturation of the modeling task is greater than or equal to the saturation threshold; d6: If, under the premise of meeting the model quality standard, steps d1 to d4 are repeated for the rated number of times, if the condition that the saturation of the modeling task is greater than or equal to the saturation threshold is still not met, an early warning reminder is issued.