A smart campus middle platform big data sharing method and system

Abstract

The present application relates to the technical field of data transmission, in particular to a big data sharing method in a wisdom campus middle station, which collects facial images of students and parents in real time, screens relevant data according to historical facial data in a storage unit, marks suspicious frame images in the relevant data to train generated simulation data, and restores facial expressions according to simulation data and displays images, which can avoid limitations and dependence of big data in propagation due to bandwidth and network problems, retain more key facial information, ensure convergence speed when training a big data model, and improve communication efficiency.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, specifically to a method and system for sharing big data in a smart campus platform. Background Technology

[0002] Existing smart campus platforms mostly utilize apps, communication communities, or telephone for parent-teacher communication. However, these methods suffer from fragmented information and short-term data transmission. Furthermore, most parents rarely notice messages or emails. Traditional telephone communication can only transmit language and cannot express facial expressions, which convey a wealth of information, leading to low communication efficiency. To address these issues, some platforms use virtual technology to collect and visualize parents' facial information. To achieve better communication, they often transmit all data in real-time. However, real-time transmission of all image data consumes significant network resources, causing congestion, and is susceptible to network fluctuations. The problem is that video stuttering can affect communication efficiency. A Chinese invention published on November 21, 2023, with announcement number CN116433432B, entitled "A Smart Campus Management System Based on Big Data," uses different sensors to acquire various feature data, compares them with feature data stored in historical databases to filter out important features, and finally uses a reconstruction and display module to complete the reconstruction of important facial expressions. This can reduce the resource consumption of real-time transmission and greatly reduce the impact of network fluctuations. Although the above method can enhance the anti-interference ability during transmission, when dealing with multi-frame images, the above screening and reconstruction method will lead to discontinuous reconstruction results, affecting the communication experience of parents and teachers. Summary of the Invention

[0003] The purpose of this invention is to propose a smart campus big data sharing method and system to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0004] To achieve the above objectives, according to one aspect of the present invention, a method for sharing big data in a smart campus platform is provided, the method comprising the following steps:

[0005] S100, collects facial images of parents as facial feature data;

[0006] S200 generates training data based on the collected facial feature data;

[0007] S300 generates facial simulation data based on training data;

[0008] S400 restores the current parent's facial expression based on facial simulation data.

[0009] Furthermore, in S300, facial simulation data is generated based on the training data by restoring the model.

[0010] Furthermore, the model can be any one of FaceNet, VGG-Face, ArcFace, or MTCNN.

[0011] Furthermore, in S400, the current parent's facial expression is restored based on facial simulation data using an imaging model.

[0012] Furthermore, the imaging model is the StyleGAN model.

[0013] Furthermore, in S100, the specific method for collecting the parent's facial image as facial feature data is as follows: Collect the parent's facial feature image according to a preset cycle and use it as the facial feature data of the parent being collected. Let i be the sequence number of the collection cycle, and B... i This represents the facial feature data collected in the i-th acquisition cycle, consisting of all B... i This forms set B.

[0014] In real life, facial expressions typically change dynamically over time. The current expression is influenced by the dynamic sequence of context in which it occurs. For example, the expressions parents and teachers display during phone conversations are also influenced by their expressions from the previous moment. The previous moment provides a contextual basis for generating a filter for the current expression. The current expression is constructed based on the emotional characteristics of this filter, causing it to shift in the same direction as the expression from the previous moment. This indicates a certain correlation between expressions. A Chinese invention published on November 21, 2023, with announcement number CN116433432B, entitled "A Smart Campus Management System Based on Big Data," illustrates this. By continuously collecting various feature data of parents through devices, filtering them to obtain feature data more similar to those in the historical storage unit, and then using the model to reconstruct the current expression, the training speed of the model can be accelerated. It can also prioritize the transmission of important expressions to ensure smooth communication in the event of network fluctuations. However, the above method only uses the current image as the basis for evaluating the current parent's emotion and does not consider the correlation between expressions. It cannot capture the weak feature data of different emotional transition periods in time, so many feature data that can represent emotional transitions are lost. This leads to incomplete model training and makes it impossible for the model to accurately reconstruct the parent's facial image. Therefore, the following method of this application is needed to obtain complete training data.

[0015] Furthermore, in S200, the method for generating training data based on the collected facial feature data includes the following steps:

[0016] S210, Generate a representative component table based on the acquired facial feature data;

[0017] S220, calculate the correlation coefficient between facial feature data in the representative component table;

[0018] S230: The training data is labeled according to the correlation coefficient and stored in the database.

[0019] Furthermore, in S210, the specific method for generating a representative component table based on the acquired facial feature data is as follows: within the range of values ​​for i, each B... i The matching degree is calculated with the pre-stored facial feature data of the collected parents. All facial feature data with a matching degree greater than the average value are marked as relevant data, and others are marked as irrelevant data. All relevant data are stored in the storage unit. In the storage unit, newly added data is marked as incremental data, and others are marked as original data. At the same time, relevant data and irrelevant data are used as representative component tables.

[0020] Furthermore, the matching degree calculation algorithm can be either Meanshift or Camshift.

[0021] Furthermore, in S220, the method for calculating the correlation coefficient between facial feature data in the representative component table is as follows: in the representative component table, the ratio of the number of related data in the same period to the total number of data in the set composed of facial feature data in the same period is used as the performance probability P of the facial feature data in the current period. j , j represents the sequence number of the acquisition period, and P represents the probability of facial feature data across all periods. j Form a set P;

[0022] All acquisition cycles are combined into a set H. The discrete coefficients of the facial feature data of the component table are calculated. The calculation method is to expand the representative component table to generate a component table. Specifically, the data in the current representative component table is added to the original representative component table after affine transformation to generate a component table. The ratio between the number of incremental data in the storage unit and the number of original data is used as a reference coefficient.

[0023] The absolute difference between the performance probability and the mean of all periods in set H is summed and averaged to obtain the discrete coefficient of the current subscale. If the discrete coefficient is greater than the reference coefficient, the period with the largest absolute difference between the performance probability and the reference probability is removed from set H. The discrete coefficients of the remaining periods in set H are then calculated again. The discrete coefficients are compared with the reference coefficients again. The above operation is repeated until the discrete coefficient is less than the reference coefficient. The performance probabilities of the remaining periods in set H are summed and averaged to obtain the discrete standard HT.

[0024] In set H, the correlation coefficient is calculated based on the discrete standard of facial feature data. The calculation method is as follows: in set H, all periods with a discrete coefficient greater than HT are selected and marked as important periods. All important periods are combined into an important set E. The pre-stored parent facial feature data is called historical data. For each period in set E, the image with the lowest matching degree with historical data is called the trough image, and the image with the highest matching degree with historical data is called the peak image.

[0025] Starting from the first period of set E, feature point detection is performed on the region of interest of the valley image to obtain multiple feature points; all the collected feature points Sr are arranged in the order of detection time to form the starting point set Start, where r represents the index of the feature point, Sr represents the r-th feature point, and r∈[1,N], N represents the number of feature points;

[0026] Feature point detection is performed on the region of interest of the peak image to obtain multiple feature points; all the collected feature points Pr are arranged in the order of detection time to form the peak point set Peak, where Pr is represented as the r-th feature point;

[0027] A reference coordinate system is established with the geometric center of the valley image as the origin. The points in the starting set and the peak set are matched one-to-one to generate direction vectors as reference vectors to form a reference set. The reference set is updated according to the time order in which the feature points in the Start set are detected.

[0028] Sequentially acquire the feature points of the regions of interest (ROIs) of all images (excluding trough and peak images) acquired in each period of set E, and mark all feature points as 01; generate a feature sequence for each frame of image based on the detected feature points in the order of detection time; starting from the first feature sequence, sequentially form a direction vector with the feature points of the acquired image and the corresponding feature points in the peak image as the change vector, and then calculate the cosine value of the angle between the change vector and the corresponding reference vector in the reference set; if the cosine value is less than zero, change the current feature point mark to 00; complete the marking of all feature points in the above manner.

[0029] The reference vector represents the changing trend of feature points in the region of interest (ROI) of the trough image and the peak image. The trough image represents the starting point of the expression in the current cycle, and the peak image represents the ending point of the expression in the current cycle. The change vector represents the changing trend of feature points in the ROI of the current image and the peak image in the current cycle. The cosine of the angle between the change vector and the reference vector reflects the direction of the changing trend. If the cosine value is less than zero, it proves that the current feature point differs significantly from the overall changing trend of the expression in the current cycle, and the image where the current feature point is located can be considered a transitional image that can represent the change of expression in the current cycle. If the cosine value is greater than zero, it proves that the current feature point differs little from the overall changing trend of the expression in the current cycle, and the image where the current feature point is located can be considered a normal image in the current cycle.

[0030] Furthermore, feature point detection is performed using the shape_predictor function from the DLIB open-source library.

[0031] Furthermore, using the face detection technology provided by DLIB, the nose region, mouth region, left eyebrow and right eyebrow region are marked as regions of interest.

[0032] The system sequentially performs non-zero checks on all feature points in all marked images (i.e., sequentially checks whether the label of all feature points in the image frame is 0 or 1), and performs a bitwise AND operation on the results. If the result is true (i.e., if all feature points are 0 or 1), the current image is marked as a normal image; if the result is false, the current image is marked as a suspicious image.

[0033] Furthermore, all periods containing suspicious images are marked as suspicious periods (Sp). k k represents the index of the suspected period, and the period Sp is denoted as . k The number of images in the database is Num, according to the formula CL=P j ×Num α Calculate Sp k The correlation coefficient, where α is the degree factor, is calculated using the formula α = exp( ), where the function exp() is an exponential function with the natural constant e as the base, and Nnum is the total number of images in all suspicious periods. Suspicious periods with a correlation coefficient greater than the average correlation coefficient are marked as associated periods;

[0034] Furthermore, the images contained in the associated period are labeled as training data and stored in the database.

[0035] The beneficial effects are as follows: The above method can filter out images with more concentrated features, thereby reducing the errors introduced by other redundant images. The trough image and the peak image represent the most subdued and most intense degree of the current expression (subdued expression means the current image has a shallow degree of expression, and intense expression means the current image has a deep degree of expression). The reference vector is a direction vector formed by using the feature points in the trough image as the starting point and the corresponding feature points in the peak image as the ending point. It can better illustrate the overall trend of the muscles in the current key facial area. When dealing with multiple frames of images, frame-by-frame detection can discover more subtle changes in expression. By calculating the correlation coefficient between images, it can accurately... It accurately captures suspicious image frames of facial expression changes, which can show the transition state of facial expression transformation, making the output of the reconstruction model smoother. At the same time, by filtering data by performance probability, redundant data can be removed, ensuring the model's superior training speed. The component table and discrete standard can reduce the amount of training data while ensuring the quality of training data, reducing the resource consumption during data transmission. It solves the problem that some servers are extremely susceptible to network fluctuations due to insufficient bandwidth, and can maintain the stability of the replicated facial expressions even under network fluctuations, enhancing the communication experience for parents and teachers.

[0036] However, current research indicates that the facial expressions presented first can influence an individual's perception of subsequent facial expressions through adaptation effects. After prolonged exposure to a specific emotional face, habitual neural responses inhibit an individual's behavioral response to that emotional face, leading to a shift in the perception of subsequent emotional faces towards the opposite emotional characteristics. For example, during a prolonged conversation between a teacher and a parent, a teacher may observe a prolonged expression of pleasure. When the teacher stares at the pleasant face for an extended period, the characteristics of the pleasant emotion become less positive, causing the perception of the current expression to shift towards a negative or indifferent one. This results in a negative transformation of the presented expression (simply put, staring at a pleasant expression for a long time may result in an unpleasant expression). In model training, these expressions that produce negative transformations can significantly interfere with the training results, greatly reducing the accuracy of the replicated model. To address these issues, this invention proposes the following method: labeling images with similar feelings to resolve the transmission interference caused by negative transformations.

[0037] In all association cycles, the first suspicious image is marked as the starting frame image. Feature point detection is performed on the region of interest of the starting frame image to obtain multiple feature points. All feature points Tp collected on the current image are arranged in the order of detection time to form the starting point set TP, where q represents the index of the feature point, Tp q This represents the q-th feature point;

[0038] Within the associated period of the starting frame image, the image with the highest matching degree with historical data is retrieved to form the local peak image. Feature point detection is performed on the region of interest (ROI) of the local peak image to obtain multiple feature points. All feature points Hp collected on the current image are arranged in the order of detection time to form the starting point set HP, where q represents the index of the feature point. q This represents the q-th feature point;

[0039] A reference coordinate system is established with the geometric center of the starting frame image as the origin. The feature points in set TP and set HP are matched one-to-one to generate direction vectors as local reference vectors to form a local reference set. The local reference set is updated according to the time order in which the feature points in set TP are detected.

[0040] Starting from the frame following the initial frame, feature point detection is performed sequentially on the region of interest (ROI) of the image in the current association period to obtain multiple feature points in the image, and all feature points are marked as 01. For each frame, the detected feature points generate a feature sequence according to the detection time order. Starting from the first feature sequence, the direction vectors formed by the collected feature points and the corresponding feature points collected in the peak image are used as local change vectors. Then, the cosine value of the angle formed between the local change vector and the corresponding local reference vector in the local reference set is calculated. If the cosine value is greater than zero, the feature point marking remains unchanged; otherwise, the current feature point marking is changed to 00. The marking of all feature points is completed in the above manner.

[0041] Since negative transitions manifest as abrupt changes in the current expression, the cosine of the angle between the local change vector and the local reference vector in the current period can indicate whether a negative transition has occurred. If the cosine value is greater than zero, it means that no negative transition has occurred in the current image; if the cosine value is less than zero, it is considered that a negative transition has occurred. By marking suspicious images as the starting frame, the interference of suspicious frame images on the calculation results is eliminated, thus enabling the accurate capture of images exhibiting negative transitions.

[0042] For all images in the associated period, the marked feature points are checked for non-zero values. Then, a bitwise AND operation is performed on the results. If the result is true, all images in the associated period between the starting frame image and the local peak image (excluding the starting frame image and the local peak image) are marked as desensitized images, and all the desensitized images are combined into a desensitized sequence List. If the result is false, the original markings are kept unchanged, and the first suspicious image in the next associated period is marked as the starting image. The above operation is repeated until all images in the associated period are marked.

[0043] In the association period, the number of images contained in the first desensitization sequence is obtained. Then, the same number of images are retrieved from the first frame of the current desensitization sequence to form a replacement sequence. All images in the current desensitization sequence are replaced with images in the replacement sequence in turn. The replacement of all desensitization sequences is completed in the above way, and the association period is updated.

[0044] The beneficial effects are as follows: Since the fading image is a negative transition phenomenon caused by a prolonged expression, replacing it with an adjacent image can eliminate transmission interference and data errors caused by the negative transition phenomenon during model training, thereby improving the accuracy of the model in replicating expressions. Since no additional processing is performed on the fading image, there is no unnecessary resource consumption, ensuring transmission speed and quality under network fluctuations. It can accurately and quickly replicate the parent's current real expression, avoid communication misunderstandings caused by the negative transition phenomenon, and enhance the communication experience for parents and schools.

[0045] Furthermore, in S300, the method for generating facial simulation data based on training data is as follows: inputting the facial data stored in the database into the restoration model to obtain pre-simulated data of the facial features of the current parent.

[0046] Furthermore, in S400, the method for restoring the current parent's facial expression based on facial simulation data is as follows: after obtaining the pre-simulation data of the current parent's facial features from the database, a virtual avatar of a specific gender is generated based on the parent's gender, and the current facial expression of the parent is restored based on the pre-simulation data of the current parent's facial features and displayed on the face of the virtual avatar corresponding to the parent.

[0047] This invention also provides a smart campus central platform big data sharing system, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program in the following system units:

[0048] The data acquisition unit is used to acquire the facial feature data of parents;

[0049] The data filtering unit is used to filter training data from facial feature images;

[0050] Data storage unit, used to store historical data and selected training data.

[0051] The model training unit is used to train the data in the storage unit to obtain facial simulation information;

[0052] The facial reconstruction unit is used to reconstruct the current facial expression of the parent based on facial simulation information.

[0053] The beneficial effects of this invention are as follows: By acquiring facial image datasets in real time and filtering relevant data from the dataset, this invention detects the movement direction and range of feature points in the regions of interest of all relevant data frame by frame. It can keenly capture subtle changes in facial expressions caused by emotional shifts during communication. This avoids the display lag problem caused by real-time transmission of all data during network fluctuations, and at the same time, it can retain more emotional information during transmission to improve communication efficiency. In the training of big data models, due to the retention of more details, it enhances the generalization ability of the model while accelerating the training speed. Attached Figure Description

[0054] Figure 1 The diagram shows a flowchart of a big data sharing method for a smart campus middleware platform.

[0055] Figure 2 The diagram shows a flowchart of a smart campus middleware big data sharing system. Detailed Implementation

[0056] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0057] Example 1: Figure 1 The diagram shows a flowchart of a big data sharing method for a smart campus platform.

[0058] Reference Figure 1 This invention proposes a method for sharing big data in a smart campus platform, the method comprising the following steps:

[0059] S100, collects facial images of parents as facial feature data;

[0060] S200 generates training data based on the collected facial feature data;

[0061] S300 generates facial simulation data based on training data;

[0062] S400 restores the current parent's facial expression based on facial simulation data.

[0063] Furthermore, in S300, facial simulation data is generated based on the training data by restoring the model.

[0064] Furthermore, the model can be any one of FaceNet, VGG-Face, ArcFace, or MTCNN.

[0065] Furthermore, the model is restored to FaceNet.

[0066] Furthermore, the facial expressions of the parents can be reconstructed using an imaging model based on facial simulation data.

[0067] Furthermore, the imaging model is the StyleGAN model.

[0068] Furthermore, in S100, the specific method for collecting the parent's facial image as facial feature data is as follows:

[0069] During the call, facial feature images of the parents are collected in 1-minute intervals and used as the facial feature data of the parents being collected. Let i be the sequence number of the collection interval, and B be the sequence number of the collection interval. i This represents the facial feature data collected in the i-th acquisition cycle, consisting of all B... i Form set B

[0070] Furthermore, in S200, generating training data based on the collected facial feature data includes the following steps:

[0071] S210, Generate a representative component table based on the acquired facial feature data;

[0072] S220, calculate the correlation coefficient between facial feature data in the representative component table;

[0073] S230: The training data is labeled according to the correlation coefficient and stored in the database.

[0074] Furthermore, in S210, the specific method for generating a representative component table based on the acquired facial feature data is as follows: within the range of values ​​for i, each B... i The matching degree is calculated with the pre-stored facial feature data of the collected parents. All facial feature data with a matching degree greater than the average value are marked as relevant data, and others are marked as irrelevant data. All relevant data are stored in the storage unit. In the storage unit, newly added data is marked as incremental data, and others are marked as original data. At the same time, relevant data and irrelevant data are used as representative component tables.

[0075] Furthermore, the matching degree calculation algorithm is the Meanshift algorithm.

[0076] Furthermore, in S220, the method for calculating the correlation coefficient between facial feature data in the representative component table is as follows: in the representative component table, the ratio of the number of related data in the same period to the total number of data in the set composed of facial feature data in the same period is used as the performance probability P of the facial feature data in the current period. j , j represents the sequence number of the acquisition period, and P represents the probability of facial feature data across all periods. jForm a set P;

[0077] All acquisition cycles are combined into a set H. The discrete coefficients of the facial feature data of the component table are calculated. The calculation method is to expand the representative component table to generate a component table. Specifically, the data in the current representative component table is added to the original representative component table after affine transformation to generate a component table. The ratio between the number of incremental data in the storage unit and the number of original data is used as a reference coefficient.

[0078] The absolute difference between the performance probability and the mean of all periods in set H is summed and averaged to obtain the discrete coefficient of the current subscale. If the discrete coefficient is greater than the reference coefficient, the period with the largest absolute difference between the performance probability and the reference probability is removed from set H. The discrete coefficients of the remaining periods in set H are then calculated again. The discrete coefficients are compared with the reference coefficients again. The above operation is repeated until the discrete coefficient is less than the reference coefficient. The performance probabilities of the remaining periods in set H are summed and averaged to obtain the discrete standard HT.

[0079] In set H, the correlation coefficient is calculated based on the discrete standard of facial feature data. The calculation method is as follows: in set H, all periods with a discrete coefficient greater than HT are selected and marked as important periods. All important periods are combined into an important set E. The pre-stored parent facial feature data is called historical data. For each period in set E, the image with the lowest matching degree with historical data is called the trough image, and the image with the highest matching degree with historical data is called the peak image.

[0080] Starting from the first period of set E, feature point detection is performed on the region of interest of the valley image to obtain multiple feature points; all the collected feature points Sr are arranged in the order of detection time to form the starting point set Start, where r represents the index of the feature point, Sr represents the r-th feature point, and r∈[1,N], N represents the number of feature points;

[0081] Feature point detection is performed on the region of interest of the peak image to obtain multiple feature points; all the collected feature points Pr are arranged in the order of detection time to form the peak point set Peak, where Pr is represented as the r-th feature point;

[0082] A reference coordinate system is established with the geometric center of the valley image as the origin. The points in the starting set and the peak set are matched one-to-one to generate direction vectors as reference vectors to form a reference set. The reference set is updated according to the time order in which the feature points in the Start set are detected.

[0083] Sequentially acquire the feature points of the regions of interest (ROIs) of all images (excluding trough and peak images) acquired in each period of set E, and mark all feature points as 01; generate a feature sequence for each frame of image based on the detected feature points in the order of detection time; starting from the first feature sequence, sequentially form a direction vector with the feature points of the acquired image and the corresponding feature points in the peak image as the change vector, and then calculate the cosine value of the angle between the change vector and the corresponding reference vector in the reference set; if the cosine value is less than zero, change the current feature point mark to 00; complete the marking of all feature points in the above manner.

[0084] Furthermore, feature point detection is performed using the shape_predictor function from the DLIB open-source library.

[0085] Furthermore, using the face detection technology provided by DLIB, the nose region, mouth region, left eyebrow and right eyebrow region are marked as regions of interest.

[0086] The system sequentially performs non-zero checks on all feature points in all marked images (i.e., sequentially checks whether the label of all feature points in the image frame is 0 or 1), and performs a bitwise AND operation on the results. If the result is true (i.e., if all feature points are 0 or 1), the current image is marked as a normal image; if the result is false, the current image is marked as a suspicious image.

[0087] Furthermore, all periods containing suspicious images are marked as suspicious periods (Sp). k k represents the index of the suspected period, and the period Sp is denoted as . k The number of images in the database is Num, according to the formula CL=P j ×Num α Calculate Sp k The correlation coefficient, where α is the degree factor, is calculated using the formula α = exp( ), where the function exp() is an exponential function with the natural constant e as the base, and Nnum is the total number of images in all suspicious periods. Suspicious periods with a correlation coefficient greater than the average correlation coefficient are marked as associated periods;

[0088] Furthermore, the images contained in the associated period are labeled as training data and stored in the database.

[0089] Furthermore, in S300, the method for generating facial simulation data based on training data is as follows: inputting the facial data stored in the database into the restoration model to obtain facial simulation data of the facial features of the current parent.

[0090] Furthermore, in S400, the method for restoring the current parent's facial expression based on facial simulation data is as follows: after obtaining the facial simulation data of the current parent's facial features from the database, a virtual avatar of a specific gender is generated based on the parent's gender, and the current facial expression of the parent is restored based on the pre-simulation data of the current parent's facial features and displayed on the face of the virtual avatar corresponding to the parent.

[0091] Example 2: The present invention also proposes the following method to solve transmission interference caused by negative conversion by marking the same-sensory image:

[0092] In all association cycles, the first suspicious image is marked as the starting frame image. Feature point detection is performed on the region of interest of the starting frame image to obtain multiple feature points. All feature points Tp collected on the current image are arranged in the order of detection time to form the starting point set TP, where q represents the index of the feature point, Tp q This represents the q-th feature point;

[0093] Within the associated period of the starting frame image, the image with the highest matching degree with historical data is retrieved to form the local peak image. Feature point detection is performed on the region of interest (ROI) of the local peak image to obtain multiple feature points. All feature points Hp collected on the current image are arranged in the order of detection time to form the starting point set HP, where q represents the index of the feature point. q This represents the q-th feature point;

[0094] A reference coordinate system is established with the geometric center of the starting frame image as the origin. The feature points in set TP and set HP are matched one-to-one to generate direction vectors as local reference vectors to form a local reference set. The local reference set is updated according to the time order in which the feature points in set TP are detected.

[0095] Starting from the frame following the initial frame, feature point detection is performed sequentially on the region of interest (ROI) of the image in the current association period to obtain multiple feature points in the image, and all feature points are marked as 01. For each frame, the detected feature points generate a feature sequence according to the detection time order. Starting from the first feature sequence, the direction vectors formed by the collected feature points and the corresponding feature points collected in the peak image are used as local change vectors. Then, the cosine value of the angle formed between the local change vector and the corresponding local reference vector in the local reference set is calculated. If the cosine value is greater than zero, the feature point marking remains unchanged; otherwise, the current feature point marking is changed to 00. The marking of all feature points is completed in the above manner.

[0096] For all images in the associated period, the marked feature points are checked for non-zero values. Then, a bitwise AND operation is performed on the results. If the result is true, all images in the associated period between the starting frame image and the local peak image (excluding the starting frame image and the local peak image) are marked as desensitized images, and all the desensitized images are combined into a desensitized sequence List. If the result is false, the original markings are kept unchanged, and the first suspicious image in the next associated period is marked as the starting image. The above operation is repeated until all images in the associated period are marked.

[0097] In the association period, the number of images contained in the first desensitization sequence is obtained. Then, the same number of images are retrieved from the first frame of the current desensitization sequence to form a replacement sequence. All images in the current desensitization sequence are replaced with images in the replacement sequence in turn. The replacement of all desensitization sequences is completed in the above way, and the association period is updated.

[0098] Furthermore, the present invention also provides an embodiment of a smart campus middleware big data sharing system, such as... Figure 2 The diagram shows a structure of a smart campus big data sharing system according to the present invention. This embodiment of the smart campus big data sharing system includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps described in the above embodiment of the smart campus big data sharing system.

[0099] The system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program in units of the following system:

[0100] The data acquisition unit is used to acquire the facial feature data of parents;

[0101] The data filtering unit is used to filter training data from facial feature images;

[0102] Data storage unit, used to store historical data and selected training data.

[0103] The model training unit is used to train the data in the storage unit to obtain facial simulation information;

[0104] The facial reconstruction unit is used to reconstruct the current facial expression of the parent based on facial simulation information.

[0105] The aforementioned smart campus big data sharing system can run on computing devices such as desktop computers, laptops, handheld computers, and cloud servers. The system that can run on this smart campus big data sharing system may include, but is not limited to, processors and memory. Those skilled in the art will understand that the examples described are merely illustrations of a smart campus big data sharing system and do not constitute a limitation on such a system. It may include more or fewer components, or combinations of certain components, or different components. For example, the smart campus big data sharing system may also include input / output devices, network access devices, buses, etc.

[0106] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the smart campus big data sharing system, connecting various parts of the system via various interfaces and lines.

[0107] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the smart campus big data sharing system by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0108] Although the invention has been described in considerable detail and particularly with regard to several of the described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiment, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

Claims

1. A method for sharing big data in a smart campus platform, characterized in that, The method includes the following steps: S100, collects facial images of parents as facial feature data; S200 generates training data based on the collected facial feature data; S300 generates facial simulation data based on training data; S400 restores the current parent's facial expression based on facial simulation data; In S200, the method for generating training data based on the collected facial feature data includes the following steps: S210, Generate a representative component table based on the acquired facial feature data; S220, calculate the correlation coefficient between facial feature data in the representative component table; In S220, the method for calculating the correlation coefficient between facial feature data in the representative component table includes: in the representative component table, calculating the ratio of the number of related data in the same period to the total number of data in the set of facial feature data in the same period as the performance probability P of the facial feature data in the current period. j j represents the sequence number of the acquisition cycle; All acquisition cycles are combined into a set H. The discrete coefficients of the facial feature data of the subscale are calculated. All cycles with discrete coefficients greater than the discrete standard HT in set H are selected and marked as important cycles. All important cycles are combined into an important set E. The pre-stored parent facial feature data is called historical data. For each cycle in set E, the image with the lowest matching degree with historical data is called the trough image, and the image with the highest matching degree with historical data is called the peak image. Starting from the first period of set E, feature point detection is performed on the region of interest of the valley image to obtain multiple feature points; all the collected feature points Sr are arranged in the order of detection time to form the starting point set Start, where r represents the index of the feature point, Sr represents the r-th feature point, and r∈[1,N], N represents the number of feature points; Feature point detection is performed on the region of interest of the peak image to obtain multiple feature points; all the collected feature points Pr are arranged in the order of detection time to form the peak point set Peak, where Pr is represented as the r-th feature point; A reference coordinate system is established with the geometric center of the valley image as the origin. The points in the starting set and the peak set are matched one-to-one to generate direction vectors as reference vectors to form a reference set. The reference set is updated according to the time order in which the feature points in the Start set are detected. Sequentially acquire the feature points of the regions of interest (ROIs) of all images in set E that do not contain trough or peak images, and mark all feature points as 01. Generate a feature sequence for each frame of image based on the detected feature points in the order of detection time. Starting from the first feature sequence, sequentially form a direction vector with the feature points of the acquired image and the corresponding feature points in the peak image as the change vector. Then calculate the cosine of the angle between the change vector and the corresponding reference vector in the reference set. If the cosine value is less than zero, change the current feature point mark to 00. Complete the marking of all feature points in the above manner. The feature points in all the marked images are sequentially checked for non-zero values, and the results are ANDed. If the result is true, the current image is marked as a normal image; if the result is false, the current image is marked as a suspicious image. Mark all periods containing suspicious images as suspicious periods (Sp). k k represents the index of the suspected period, and the period Sp is denoted as . k The number of images in the database is Num, according to the formula CL=P j ×Num α Calculate Sp k The correlation coefficient, where α is the degree factor, is calculated using the formula α = exp( The function exp() is an exponential function with the natural constant e as the base, and Nnum is the total number of images in all suspicious periods. Suspicious periods with a correlation coefficient greater than the average correlation coefficient are marked as associated periods, and the images contained in the associated periods are marked as training data and stored in the database.

2. The method for sharing big data in a smart campus middleware platform according to claim 1, characterized in that, In S100, the method for collecting parents' facial images as facial feature data is as follows: Parents' facial feature images are collected according to a preset cycle and used as the facial feature data of the parent being collected. Let i be the sequence number of the collection cycle, and B... i This represents the facial feature data collected in the i-th acquisition cycle, consisting of all B... i This forms set B.

3. The method for sharing big data in a smart campus middleware platform according to claim 1, characterized in that, In S300, the method for generating facial simulation data based on training data is as follows: The facial data stored in the database is input into the reconstruction model to obtain facial simulation data of the current parent's facial features.

4. The method for sharing big data in a smart campus middleware platform according to claim 1, characterized in that, In S400, the method for restoring the current parent's facial expression based on facial simulation data is as follows: After retrieving facial simulation data of the current parent's facial features from the database, a virtual avatar of the specific gender is generated based on the parent's gender. The current facial expression of the parent is restored based on the pre-simulation data of the current parent's facial features and displayed on the face of the virtual avatar corresponding to the parent.

5. A smart campus middleware big data sharing system, characterized in that, The smart campus platform big data sharing system includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the smart campus platform big data sharing method according to any one of claims 1-4.

Images