A multimodal data processing method

Abstract

The present invention relates to the technical field of data processing methods, and more particularly to a multimodal data processing method, comprising the following steps: S1: a data collection module receives and processes relevant data, obtains information on math scores and physics scores, and transmits the information to a control module; S2: the control module calculates a collated math score based on the math score, calculates a collated physics score based on the physics score, calculates a correlation factor between the math score and the physics score, obtains correlation information between the math score and the physics score based on the correlation factor, and transmits the information to a communication module; S3: the communication module transmits the correlation information between the math score and the physics score to a user end. The control module can quickly determine whether two sets of data are correlated, thereby accelerating model training.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing methods, and in particular to a multimodal data processing method. Background Art

[0002] Multimodality refers to the collaborative reasoning of multiple heterogeneous modal data. It integrates or fuses two or more biometric technologies, utilizes the unique advantages of multiple biometric technologies, and combines data fusion technology to make the authentication and identification process more accurate and secure.

[0003] In the field of artificial intelligence, multimodality generally refers to the use of multiple information input methods, such as images, text, and voice, to help AI more accurately understand the external world. In daily life, we can also express ourselves through multimodality, such as through language, gestures, and facial expressions to convey our emotions and intentions.

[0004] Association model building mainly involves associating multiple data tables to discover the relationships between them.

[0005] After extensive searching and reference, we found that many training methods for association construction models have been developed. For example, the prior art method for training association construction models disclosed in publication number CN115934688A includes the following steps: data collection: collecting raw data related to the association model subject; data cleaning: processing and analyzing the collected data to eliminate errors, outliers and duplicate information; feature extraction: extracting features related to the association model from the data; model training: training the association model based on the extracted features; model prediction: using the trained association model to make predictions or decisions.

[0006] The above training process is long and complicated. When users urgently need to obtain the results of an association construction model, the existing technology cannot meet their needs. Summary of the Invention

[0007] The purpose of the present invention is to improve the processing speed. In view of the above-mentioned shortcomings, a multimodal data processing method is proposed.

[0008] The present invention adopts the following technical solutions:

[0009] A multimodal data processing method comprises the following steps:

[0010] S1: The data collection module receives and processes relevant data and obtains information on math scores and physics scores, and transmits it to the control module;

[0011] S2: The control module calculates the sorted math scores based on the math scores, calculates the sorted physics scores based on the physics scores, calculates the correlation factor between the math scores and the physics scores based on the covariance of the sorted math scores and the sorted physics scores, the standard deviation of the sorted math scores, and the standard deviation of the sorted physics scores, obtains the correlation information between the math scores and the physics scores based on the correlation factor, and transmits the information to the communication module;

[0012] S3: The communication module transmits the correlation information between the math scores and the physics scores to the user end.

[0013] Optionally, the data collection module includes a multimodal data input submodule, a data conversion submodule and a data integration submodule, wherein the data conversion submodule and the data integration submodule are both communicatively connected to the multimodal data input submodule, and the data integration submodule is communicatively connected to the control module;

[0014] The multimodal data input submodule is used to receive relevant data of image, text and voice input, transmit relevant data of image and voice input to the data conversion submodule, and transmit relevant data of text input to the data integration submodule;

[0015] The data conversion submodule is used to convert the relevant data of the image and voice input, and transmit the converted relevant data to the data integration submodule;

[0016] The data integration submodule integrates the text input related data and the converted related data, and obtains information on math scores and physics scores, and transmits the information to the control module.

[0017] Optionally, when the control module calculates the correlation factor between the math scores and the physics scores, the following formula is satisfied:

[0018]

[0019] X=[x n , x n+1 ,…,x N ];

[0020] Y=[y n ,y n+1 ,…,y N ];

[0021] ΔX=[x n *α,x n+1 *α,…,x N *α];

[0022] ΔY=[y n *α,y n+1 *α,…,y N*α];

[0023] Among them, GLX is the correlation factor between mathematics and physics scores, XFC(ΔX,ΔY) is the covariance between the adjusted mathematics and physics scores, BZC ΔX is the standard deviation of math scores after adjustment, BZC ΔY is the standard deviation of the adjusted physics scores, ΔX is the adjusted mathematics scores, and ΔY is the adjusted physics scores;

[0024] X is the math score, x n is the math score of the nth student, x N is the math score of the Nth student;

[0025] Y is the physics score, y n is the physics score of the nth student, y N is the physics score of the Nth student;

[0026] α is a conversion index, and α has the following values: α = 1 or α = 0.98 or α = 0.95. When α = 1, the student's score is input in text form, when α = 0.98, the student's score is input in voice form, and when α = 0.95, the student's score is input in image form.

[0027] Optionally, when the control module calculates the correlation information between the math scores and the physics scores, the following formula is satisfied:

[0028]

[0029] Among them, β is the correlation information between mathematics scores and physics scores, γ1 to γ4 are the selection thresholds of different correlation factors between mathematics scores and physics scores. When β = 1, it indicates that there is a negative linear relationship between mathematics scores and physics scores. When β = 2, it indicates that there is no linear relationship between mathematics scores and physics scores. When β = 3, it indicates that there is a positive linear relationship between mathematics scores and physics scores.

[0030] Optionally, in step S1, the following steps are further included:

[0031] The data collection module transmits the information of mathematics and physics scores to the data cleaning module;

[0032] The data cleaning module deletes abnormal data and missing data from the math scores, and obtains information on the total number of math scores before data cleaning, the math score of the z-th student after data cleaning, the total number of math scores after data cleaning, the total time of data cleaning, and the math score of the Z-th student after data cleaning, and transmits it to the control module;

[0033] In step S2, the following steps are also included:

[0034] The control module calculates the math score after data cleaning according to the math score of the Zth student after data cleaning and the math score of the zth student after data cleaning, calculates the data cleaning determination factor according to the total number of math scores before data cleaning, the math score of the nth student, the total number of math scores after data cleaning, and the math score of the zth student after data cleaning, obtains alarm information according to the data cleaning determination factor and transmits it to the communication module;

[0035] In step S3, the following steps are also included:

[0036] The communication module transmits the alarm information to the user end and the alarm end.

[0037] Optionally, the data cleaning module includes a data cleaning submodule and a data analysis submodule that are communicatively connected, and the data analysis submodule is communicatively connected to the control module;

[0038] The data cleaning submodule cleans the abnormal data and missing data of the mathematics scores and obtains the relevant data after data cleaning, and transmits it to the data analysis submodule;

[0039] The data analysis submodule is used to analyze and obtain information on the total number of students with math scores before data cleaning, the math score of the zth student after data cleaning, the total number of students with math scores after data cleaning, the total duration of data cleaning, and the math score of the Zth student after data cleaning, and transmit the information to the control module.

[0040] Optionally, the communication module includes an information transmission submodule and an alarm submodule that are communicatively connected, and the information transmission submodule and the alarm submodule are both communicatively connected to the control module;

[0041] The control module transmits the correlation information between the math scores and the physics scores to the information transmission submodule, and synchronously transmits the alarm information to the information transmission submodule and the alarm submodule;

[0042] The information transmission submodule transmits the correlation information between the math scores and the physics scores and the alarm information to the user end;

[0043] The alarm submodule transmits the alarm information to the alarm terminal and implements the alarm according to the alarm information.

[0044] Optionally, when the control module calculates the data cleaning determination factor, the following formula is satisfied:

[0045]

[0046] X c =[x z , xz+1 ,…,x Z ];

[0047] Among them, QX is the data cleaning determination factor, N is the total number of math scores before data cleaning, x z is the math score of the zth student after data cleaning, Z is the total number of math scores after data cleaning, and T is the total time of data cleaning;

[0048] X c is the math score after data cleaning, x Z This is the math score of the Zth student after data cleaning.

[0049] Optionally, when the control module calculates the alarm information, the following formula is satisfied:

[0050]

[0051] Among them, BJ is the alarm information, qx ref It is the selection threshold of the data cleaning judgment factor. When BJ=1, the data cleaning is normal and no alarm is needed. When BJ=2, the data cleaning is abnormal and an alarm is needed.

[0052] The beneficial effects achieved by the present invention are:

[0053] 1. The control module can quickly determine whether two sets of data are correlated. Calculating the correlation between math scores and physics scores is equivalent to quickly confirming the correlation between the two sets of data, facilitating subsequent model training operations and speeding up model training.

[0054] 2. Since data cleaning in the prior art does not set relevant cleaning requirements, after the data cleaning step, the control module calculates the alarm information to further determine whether the data cleaning is abnormal, which can improve the accuracy of the model construction;

[0055] 3. The alarm submodule can issue an alarm in real time based on the alarm information, thereby speeding up the overall response speed and enabling staff to promptly discover abnormal data cleaning problems.

[0056] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and illustration and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of the method of the present invention;

[0058] Figure 2 It is a schematic diagram of the overall structure of the present invention;

[0059] Figure 3 This is a schematic diagram of the overall structure of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0060] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual dimensions. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0061] Example 1: This example provides a multimodal data processing method, combining Figure 1 and Figure 2 shown.

[0062] A multimodal data processing method comprises the following steps:

[0063] S1: The data collection module receives and processes relevant data and obtains information on math scores and physics scores, and transmits it to the control module;

[0064] S2: The control module calculates the sorted math scores based on the math scores, calculates the sorted physics scores based on the physics scores, calculates the correlation factor between the math scores and the physics scores based on the covariance of the sorted math scores and the sorted physics scores, the standard deviation of the sorted math scores, and the standard deviation of the sorted physics scores, obtains the correlation information between the math scores and the physics scores based on the correlation factor, and transmits the information to the communication module;

[0065] S3: The communication module transmits the correlation information between the math scores and the physics scores to the user end.

[0066] Optionally, the data collection module includes a multimodal data input submodule, a data conversion submodule and a data integration submodule, wherein the data conversion submodule and the data integration submodule are both communicatively connected to the multimodal data input submodule, and the data integration submodule is communicatively connected to the control module;

[0067] The multimodal data input submodule is used to receive relevant data of image, text and voice input, transmit relevant data of image and voice input to the data conversion submodule, and transmit relevant data of text input to the data integration submodule;

[0068] The data conversion submodule is used to convert the relevant data of the image and voice input, and transmit the converted relevant data to the data integration submodule;

[0069] The data integration submodule integrates the text input related data and the converted related data, and obtains information on math scores and physics scores, and transmits the information to the control module.

[0070] Optionally, when the control module calculates the correlation factor between the math scores and the physics scores, the following formula is satisfied:

[0071]

[0072] X=[x n , x n+1 ,…,x N ];

[0073] Y=[y n ,y n+1 ,…,y N ];

[0074] ΔX=[x n *α,x n+1 *α,…,x N *α];

[0075] ΔY=[y n *α,y n+1 *α,…,y N *α];

[0076] Among them, GLX is the correlation factor between mathematics and physics scores, XFC(ΔX,ΔY) is the covariance between the adjusted mathematics and physics scores, BZC ΔX is the standard deviation of math scores after adjustment, BZC ΔY is the standard deviation of the adjusted physics scores, ΔX is the adjusted mathematics scores, and ΔY is the adjusted physics scores;

[0077] X is the math score, x n is the math score of the nth student, x N is the math score of the Nth student;

[0078] Y is the physics score, y n is the physics score of the nth student, y N is the physics score of the Nth student;

[0079] α is a conversion index, and α has the following values: α = 1 or α = 0.98 or α = 0.95. When α = 1, the student's score is input in text form, when α = 0.98, the student's score is input in voice form, and when α = 0.95, the student's score is input in image form.

[0080] Specifically, the following matters need to be noted when calculating the conversion index: inputting students' scores in text form refers to the use of text input when entering the data input system, that is, the system directly outputs students' scores during online grading, and students' scores are input in text form at this time; inputting students' scores in voice form refers to the teacher directly inputting students' scores by voice when entering the data input system, and voice needs to be converted into text at this time; inputting students' scores in image form refers to the teacher taking a picture of the test paper score when entering the data input system, and images need to be converted into text at this time. Since there are certain errors in the voice and image conversion methods, setting the conversion index is equivalent to the conversion accuracy.

[0081] The following specific numbers are used to illustrate how the correlation factors of math scores and physics scores are calculated. Assuming the math scores are "80, 85, 90, 75, 100" and the physics scores are "70, 75, 85, 60, 90", the corresponding math scores and physics formulas are:

[0082] X = [80, 85, 90, 75, 100];

[0083] Y=[70,75,85,60,90];

[0084] Assume that the score of the fifth student in the math score is input as an image, the scores of the remaining students are input as text, and the scores of all students in the physics score are input as text. The corresponding formulas for the sorted math scores and sorted physics scores are:

[0085] ΔX=[80*1,85*1,90*1,75*1,100*0.95]=[80,85,90,75,95];

[0086] ΔY=[70*1,75*1,85*1,60*1,90*1]=[70,75,85,60,90];

[0087] Again

[0088] Again

[0089] Then calculate the covariance of the sorted math scores and the sorted physics scores, which is the following formula:

[0090]

[0091] Then calculate the standard deviation of the math scores after sorting, which is the following formula:

[0092]

[0093] Then calculate the standard deviation of the physics scores after sorting, which is the following formula:

[0094]

[0095] Then calculate the correlation factor between mathematics scores and physics scores, which is the following formula:

[0096]

[0097] Optionally, when the control module calculates the correlation information between the math scores and the physics scores, the following formula is satisfied:

[0098]

[0099] Among them, β is the correlation information between mathematics scores and physics scores, γ1 to γ4 are the selection thresholds of different correlation factors between mathematics scores and physics scores. When β = 1, it indicates that there is a negative linear relationship between mathematics scores and physics scores. When β = 2, it indicates that there is no linear relationship between mathematics scores and physics scores. When β = 3, it indicates that there is a positive linear relationship between mathematics scores and physics scores.

[0100] Preferably, specific numbers are listed below to illustrate how to calculate the correlation information between math scores and physics scores, such as the following formula:

[0101]

[0102] It can be further set that when the correlation factor between math scores and physics scores is closer to -1.25, the negative linear relationship is more obvious; when the correlation factor between math scores and physics scores is closer to 0.75, the positive linear relationship is more obvious. When applied to other embodiments, the selection threshold of the corresponding correlation factor between math scores and physics scores will be appropriately changed.

[0103] Optionally, in step S1, the following steps are further included:

[0104] The data collection module transmits the information of mathematics and physics scores to the data cleaning module;

[0105] The data cleaning module deletes abnormal data and missing data from the math scores, and obtains information on the total number of math scores before data cleaning, the math score of the z-th student after data cleaning, the total number of math scores after data cleaning, the total time of data cleaning, and the math score of the Z-th student after data cleaning, and transmits it to the control module;

[0106] In step S2, the following steps are also included:

[0107] The control module calculates the math score after data cleaning according to the math score of the Zth student after data cleaning and the math score of the zth student after data cleaning, calculates the data cleaning determination factor according to the total number of math scores before data cleaning, the math score of the nth student, the total number of math scores after data cleaning, and the math score of the zth student after data cleaning, obtains alarm information according to the data cleaning determination factor and transmits it to the communication module;

[0108] In step S3, the following steps are also included:

[0109] The communication module transmits the alarm information to the user end and the alarm end.

[0110] Optionally, the data cleaning module includes a data cleaning submodule and a data analysis submodule that are communicatively connected, and the data analysis submodule is communicatively connected to the control module;

[0111] The data cleaning submodule cleans the abnormal data and missing data of the mathematics scores and obtains the relevant data after data cleaning, and transmits it to the data analysis submodule;

[0112] The data analysis submodule is used to analyze and obtain information on the total number of students with math scores before data cleaning, the math score of the zth student after data cleaning, the total number of students with math scores after data cleaning, the total duration of data cleaning, and the math score of the Zth student after data cleaning, and transmit the information to the control module.

[0113] Optionally, the communication module includes an information transmission submodule and an alarm submodule that are communicatively connected, and the information transmission submodule and the alarm submodule are both communicatively connected to the control module;

[0114] The control module transmits the correlation information between the math scores and the physics scores to the information transmission submodule, and synchronously transmits the alarm information to the information transmission submodule and the alarm submodule;

[0115] The information transmission submodule transmits the correlation information between the math scores and the physics scores and the alarm information to the user end;

[0116] The alarm submodule transmits the alarm information to the alarm terminal and implements the alarm according to the alarm information.

[0117] Optionally, when the control module calculates the data cleaning determination factor, the following formula is satisfied:

[0118]

[0119] X c =[x z , x z+1 ,…,x Z ];

[0120] Among them, QX is the data cleaning determination factor, N is the total number of math scores before data cleaning, x z is the math score of the zth student after data cleaning, Z is the total number of math scores after data cleaning, and T is the total time of data cleaning;

[0121] X c is the math score after data cleaning, x z This is the math score of the Zth student after data cleaning.

[0122] Specifically, data cleaning refers to deleting abnormal data and missing data. Abnormal data refers to data that is extreme or inconsistent with common sense, and missing data refers to data that was not filled in during input. The following uses specific values ​​to illustrate how to delete abnormal data. When entering math scores, it is known that the full score is 120, a's math score is 50, b's math score is 60, c's math score is not entered, d's math score is 80, e's math score is 130, and f's math score is D. The above integration in the formula is as follows:

[0123] X = [50, 60, not input, 80, 130, D level];

[0124] The corresponding "not entered" is classified as missing data, and "130" and "Grade D" are abnormal data. They are integrated into the formula as follows:

[0125] X c =[50, 60, 80];

[0126] All the classified missing data and abnormal data will be deleted.

[0127] Optionally, when the control module calculates the alarm information, the following formula is satisfied:

[0128]

[0129] Among them, BJ is the alarm information, qx ref It is the selection threshold of the data cleaning judgment factor. When BJ=1, the data cleaning is normal and no alarm is needed. When BJ=2, the data cleaning is abnormal and an alarm is needed.

[0130] Although the above formula does not provide specific units in this embodiment, those skilled in the art will certainly set appropriate units according to the needs of actual applications.

[0131] This embodiment solves the problem of long processing time in the traditional training method of building a multimodal association model. Specifically, this embodiment can quickly determine whether two sets of data are correlated through the control module, which facilitates subsequent model training operations and speeds up model training.

[0132] In addition, since the data cleaning in the prior art does not set relevant cleaning requirements, after the data cleaning step, the control module calculates the alarm information to further determine whether the data cleaning is abnormal, which can improve the accuracy of the constructed model.

[0133] Finally, the alarm submodule can issue real-time alarms based on the alarm information, thereby speeding up the overall response speed and enabling staff to promptly detect data cleaning anomalies.

[0134] Example 2: This example includes all the contents of Example 1 and provides a multimodal data processing method. Figure 3 shown.

[0135] A multimodal data processing method further includes the following steps:

[0136] In step S3, the following steps are also included:

[0137] The communication module transmits the alarm information to the abnormal data acquisition module.

[0138] The abnormal data acquisition module identifies abnormal data, statistical information, and obtains the total number of historical scores, the cth historical score, the Cth historical score, and the student's leave reference index for this semester, and transmits the information to the control module;

[0139] In step S2, the following steps are also included:

[0140] The control module calculates the value of the abnormal data based on the total number of historical scores, the cth historical score, the Cth historical score, and the student's leave reference index this semester, and obtains the true value of the abnormal data based on the calculated value of the abnormal data and transmits it to the data collection module.

[0141] Specifically, when the alarm submodule implements an alarm, it indicates that the data cleaning is abnormal, that is, too much data is deleted, resulting in the overall data being not of reference value. Therefore, by adding an abnormal data acquisition module and a control module, the true value of the abnormal data is calculated, and the true value of the abnormal data is added to the data integration submodule, and then related operations are performed, which will reduce the amount of deleted data; setting the true value of the abnormal data calculated by the control module is a text input method.

[0142] Optionally, when the control module calculates the calculated value of abnormal data, the following formula is satisfied:

[0143]

[0144] Among them, YCSJ js is the calculated value of abnormal data, C is the total number of historical scores, CJ cFor the cth historical result, CJ C is the Cth historical score, CQ is the reference index of the student's leave this semester, and CQ has the following values: CQ = 0.95, CQ = 0.90, or CQ = 0.85. When CQ = 0.95, the cumulative number of days of leave for the student this semester is less than 8 days. When CQ = 0.90, the cumulative number of days of leave for the student this semester ranges from greater than or equal to 8 days to less than 15 days. When CQ = 0.85, the cumulative number of days of leave for the student this semester ranges from greater than or equal to 15 days.

[0145] Specifically, the following matters need to be noted when calculating the total number of historical scores. The historical score refers to the previous C historical scores used as reference scores when the current score is abnormal, and the historical scores and the current score are set to be the same subject. In addition, the total number of historical scores is limited to greater than or equal to 5 times, that is, C≥5. When C<5, the abnormal data is converted into missing data and the missing data is deleted.

[0146] Optionally, when the control module calculates the true value of the abnormal data, the following formula is satisfied:

[0147]

[0148] Among them, YCSJ tc is the true value of the abnormal data, and μ is the selection threshold of the score difference.

[0149] Specifically, the selection threshold of the performance difference is preset by those skilled in the art.

[0150] Although the above formula does not provide specific units in this embodiment, those skilled in the art will certainly set appropriate units according to the needs of actual applications.

[0151] After the calculation of the abnormal data is completed, the existing correlation construction model is used to analyze the correlation between the scores of different subjects. Specifically, the Spearman rank correlation coefficient is used to measure the dependence between two variables. The analysis principle is as follows: first, each variable is assigned a grade, and then the Spearman rank correlation coefficient is calculated using the formula of the Spearman correlation coefficient. For example, taking the calculation of the correlation between mathematics and physics scores as an example, the scores of mathematics and physics are first assigned grades, and the students with the highest grades are limited to grade 1. The lower the grade, the larger the grade value. Then, the Spearman rank correlation coefficient is calculated according to the above steps. When the score of one student is missing, the ranking of the student's score cannot be calculated, so the data of this student needs to be excluded during the calculation. In the prior art, the relevant algorithm is required to rank the scores, which is relatively cumbersome, and when the rankings of students are the same, it is easy to make mistakes, which will slow down the overall calculation speed. Therefore, the present invention estimates the correlation through a simple calculation method, and the user can quickly estimate the correlation. If the user needs to accurately know the degree of correlation, it can then determine whether the cleaned data is abnormal, and then perform the corresponding data supplement. Finally, the existing correlation construction model is used to analyze the correlation.

[0152] This embodiment solves the problem of incorrect data cleaning in the training method of the traditional multimodal association construction model. Specifically, this embodiment calculates the true value of the abnormal data by adding an abnormal data acquisition module and a control module, reduces the amount of deleted data, and ensures the quality of data cleaning; because the control module refers to the student's historical grades when calculating, the calculated value of the abnormal data is estimated as much as possible, and the student's attendance can be judged based on the student's leave reference index this semester, which can reflect whether the student has missed a large number of knowledge points.

[0153] The contents disclosed above are only preferred feasible embodiments of the present invention and do not limit the scope of protection of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the scope of protection of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A multimodal data processing method, characterized in that: The following steps are involved: S1: The data collection module receives and processes relevant data and obtains information on math scores and physics scores, and transmits it to the control module; S2: The control module calculates the sorted math scores based on the math scores, calculates the sorted physics scores based on the physics scores, calculates the correlation factor between the math scores and the physics scores based on the covariance of the sorted math scores and the sorted physics scores, the standard deviation of the sorted math scores, and the standard deviation of the sorted physics scores, obtains the correlation information between the math scores and the physics scores based on the correlation factor, and transmits the information to the communication module; S3: The communication module transmits the correlation information between the math scores and the physics scores to the user end; When the control module calculates the correlation factor between the math scores and the physics scores, the following formula is satisfied: ； ； ； ； ； in, is the correlation factor between mathematics and physics scores, is the covariance of the adjusted math scores and the adjusted physics scores, is the standard deviation of the math scores after adjustment, is the standard deviation of the adjusted physics scores, For the math scores after sorting, is the sorted physics score; For math scores, For the Mathematics scores of students, For the Mathematics scores of students; For physics scores, For the The physics scores of the students For the the physics scores of the students; is the conversion index, The following values ​​are respectively available: or or ,when When the student's grades are entered in text format, When the student's score is input by voice, When the student's score is input in the form of image; When the control module calculates the correlation information between the math scores and the physics scores, the following formula is satisfied: ； in, For the correlation information between math scores and physics scores, to The thresholds for selecting correlation factors for different mathematics and physics scores are as follows: When , it shows that there is a negative linear relationship between mathematics and physics scores. When , it shows that there is no linear relationship between math scores and physics scores. It shows that there is a positive linear relationship between mathematics scores and physics scores.

2. A multimodal data processing method according to claim 1, characterized in that: The data collection module includes a multimodal data input submodule, a data conversion submodule and a data integration submodule, wherein the data conversion submodule and the data integration submodule are both communicatively connected to the multimodal data input submodule, and the data integration submodule is communicatively connected to the control module; The multimodal data input submodule is used to receive relevant data of image, text and voice input, transmit relevant data of image and voice input to the data conversion submodule, and transmit relevant data of text input to the data integration submodule; The data conversion submodule is used to convert the relevant data of the image and voice input, and transmit the converted relevant data to the data integration submodule; The data integration submodule integrates the text input related data and the converted related data, and obtains information on math scores and physics scores, and transmits the information to the control module.

3. A multimodal data processing method according to claim 2, characterized in that: In step S1, the following steps are also included: The data collection module transmits the information of mathematics and physics scores to the data cleaning module; The data cleaning module deletes abnormal data and missing data on the math scores, and obtains the total number of math scores before and after data cleaning. the math scores of students, the total number of math scores after data cleaning, the total time of data cleaning and the time after data cleaning The information of the mathematics scores of each student is collected and transmitted to the control module; In step S2, the following steps are also included: The control module cleans the data The math scores of students after data cleaning Calculate the math scores of the students after data cleaning, calculate the data cleaning judgment factor according to the total number of math scores before data cleaning, the math scores of N students before data cleaning, the total number of math scores after data cleaning, and the math scores of Z students after data cleaning, and obtain the alarm information according to the data cleaning judgment factor and transmit it to the communication module; In step S3, the following steps are also included: The communication module transmits the alarm information to the user end and the alarm end.

4. A multimodal data processing method according to claim 3, characterized in that: The data cleaning module includes a data cleaning submodule and a data analysis submodule that are communicatively connected, and the data analysis submodule is communicatively connected to the control module; The data cleaning submodule cleans the abnormal data and missing data of the mathematics scores and obtains the relevant data after data cleaning, and transmits it to the data analysis submodule; The data analysis submodule is used to analyze and obtain the total number of math scores before data cleaning, the math scores of N students before data cleaning, the total number of math scores after data cleaning, the total time of data cleaning and the total time after data cleaning. The information of each student's math score is collected and transmitted to the control module.

5. A multimodal data processing method according to claim 4, characterized in that: The communication module includes an information transmission submodule and an alarm submodule that are communicatively connected, and the information transmission submodule and the alarm submodule are both communicatively connected to the control module; The control module transmits the correlation information between the math scores and the physics scores to the information transmission submodule, and synchronously transmits the alarm information to the information transmission submodule and the alarm submodule; The information transmission submodule transmits the correlation information between the math scores and the physics scores and the alarm information to the user end; The alarm submodule transmits the alarm information to the alarm terminal and implements the alarm according to the alarm information.

6. A multimodal data processing method according to claim 5, characterized in that: When the control module calculates the data cleaning determination factor, the following formula is satisfied: ； ； in, is the data cleaning determination factor, is the total number of people with the highest math scores before data cleaning, After data cleaning Mathematics scores of students, is the total number of people with math scores after data cleaning, The total time for data cleaning; For the Mathematics scores of students; is the math score after data cleaning, After data cleaning A student's math scores.

7. A multimodal data processing method according to claim 6, characterized in that: When the control module calculates the alarm information, the following formula is satisfied: ； in, For alarm information, is the selection threshold of the data cleaning decision factor, when When the data is cleaned normally, no alarm is required. When data cleaning is abnormal, an alarm is required.

Images