Big data processing method and system based on birth and education fusion

Abstract

The invention relates to the technical field of big data, in particular to a big data processing method and system based on production and teaching fusion, and the method comprises the steps: collecting real operation information of a plurality of target posts at different sampling times, and extracting keyword phrases; clustering the keyword phrases according to the sampling time to obtain core phrases; determining a fluctuation characteristic value of each core keyword; determining a deviation value of each core keyword according to the difference of each core keyword at each adjacent sampling time and the difference of the fluctuation characteristic values; determining the dispersion deviation of each core keyword based on the dispersion of the keywords obtained by multiple times of sampling and the frequency deviation of the core keyword in each time of sampling; determining a recommendation sequence of each core keyword based on the dispersion deviation and the deviation value of the core keywords, and performing subject matching according to the recommendation sequence of each core keyword to generate a recommended teaching subject; the teaching content can be dynamically adjusted according to post requirements, and the integrating degree of talent training and industrial requirements is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a big data processing method and system based on industry-education integration. Background Art

[0002] As a key pillar of the national economy, the textile and apparel industry is experiencing both challenges and opportunities brought about by technological and market changes. Intelligentization, digitalization, and green development are becoming inevitable trends in industrial transformation and upgrading, while consumers are increasingly demanding personalized and diversified products.

[0003] Industry-education integration refers to the close integration of industry and education, guiding teaching content through enterprise needs, achieving a seamless connection between theory and practice, and enhancing students' employability. The development of the textile and apparel industry requires talent across the entire industry chain who possess both technical expertise, innovation capabilities, and market sensitivity.

[0004] There have been attempts to apply big data analysis to textile and apparel teaching, but this lacks systematic integration. Therefore, it is necessary to build a comprehensive talent development system that integrates teaching, practical training, scientific research, and employment, and to achieve intelligent, precise, and efficient talent development through information technology. Summary of the Invention

[0005] To solve the above problems, the present invention provides a big data processing method and system based on the integration of industry and education, which aims to accurately match teaching content through big data analysis of the actual needs of the industry, improve the pertinence and effectiveness of talent training, and meet the demand of the textile and garment industry for compound talents.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In one aspect, an embodiment of the present invention provides a method for processing big data based on industry-education integration, the method comprising the following steps:

[0008] S100, obtaining keyword phrases corresponding to M target positions at the latest N sampling times, to obtain M×N keyword phrases; the keyword phrases contain multiple keywords; M and N are both integers greater than 1;

[0009] S200, clustering the keyword phrases corresponding to the M target positions according to the sampling time to obtain N groups of core phrases; the core phrases contain multiple core keywords;

[0010] S300, determining a fluctuation characteristic value of each core keyword based on a change trend and a distribution dispersion of the core keywords in the N groups of core phrases;

[0011] S400, determining a deviation value of each core keyword based on a difference between each core keyword at each adjacent sampling time and a difference between the fluctuation characteristic values;

[0012] S500: Sampling the total sample consisting of all keywords for all target positions multiple times, and determining the dispersion deviation of each core keyword based on the dispersion of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling;

[0013] S600: Determine the recommended ranking of each core keyword based on the discreteness deviation and the deviation value of the core keyword, perform subject matching based on the recommended ranking of each core keyword, and generate recommended teaching subjects based on the matched subjects.

[0014] Optionally, in S100, the keyword phrases corresponding to the M target positions at the latest N sampling times are obtained to obtain M×N keyword phrases, including:

[0015] S110, obtaining an industry practice database and a teaching database, wherein the industry practice database includes multiple target positions and corresponding practical operation information;

[0016] S120, extracting keywords from the practical operation information of the target position in order of sampling time to obtain N groups of position keywords;

[0017] S130 , sorting N groups of job keywords according to the time dimension to obtain a job keyword matrix.

[0018] Optionally, in S200, the keyword phrases corresponding to the M target positions are clustered according to the sampling time to obtain N groups of core phrases, including:

[0019] S210, performing frequency statistics on the keyword phrases at each sampling time to filter out high-frequency keywords;

[0020] S220, calculating the similarity between high-frequency keywords and constructing a similarity matrix; clustering the high-frequency keywords using an agglomerative hierarchical clustering algorithm based on the similarity matrix to obtain multiple high-frequency keyword sets;

[0021] S230 , grouping the high-frequency keywords in each high-frequency keyword set into a group of high-frequency phrases, and performing weight distribution on the N groups of high-frequency phrases to obtain N groups of core phrases.

[0022] Optionally, in S300, determining the fluctuation characteristic value of each core keyword based on the change trend and distribution dispersion of the core keywords in the N groups of core phrases includes:

[0023] S310, calculating the frequency of occurrence of each core keyword at each sampling time point, fitting a curve showing the change of frequency over time, and using the slope of the curve as the change trend of the core keyword;

[0024] S320, obtaining the distribution dispersion of the core keyword based on the variance of the frequency of occurrence of the core keyword at each sampling time point;

[0025] S330 , taking the product of the slope and variance of the core keyword at each sampling time point as the characteristic value of the sampling time point, and accumulating the characteristic values of each sampling time to obtain the fluctuation characteristic value of the core keyword.

[0026] Optionally, in S400, determining the deviation value of each core keyword according to the difference between each core keyword at each adjacent sampling time and the difference between the fluctuation characteristic values includes:

[0027] S410: For each core keyword, the core keyword is used as the target keyword, the core phrase containing the target keyword is used as the target phrase, the target phrases at adjacent sampling times are converted into word vector representations, and the cosine distance between the two word vectors is calculated using the cosine similarity formula as the difference between the target keywords;

[0028] S420, calculating the difference between the fluctuation characteristic values of the target keyword at adjacent sampling times, and taking the absolute value thereof as the difference in the fluctuation characteristic value of the target keyword;

[0029] S430 , according to a preset weight coefficient, weighted sums are taken of the differences of the target keyword and the differences of the fluctuation characteristic values at each sampling time, and then accumulated to obtain a deviation value of the target keyword.

[0030] Optionally, in S500, the total sample consisting of all keywords for all target positions is sampled multiple times, and based on the discreteness of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling, the discreteness deviation of each core keyword is determined, including:

[0031] S510, randomly dividing the total sample into several sub-samples, calculating the dispersion degree of the keywords in each sub-sample respectively, and taking the average of the dispersion degrees as the dispersion degree of the total sample;

[0032] S520, counting the frequency of occurrence of each core keyword in each sampling, and using the standard deviation of the frequency of occurrence in each sampling as the frequency deviation of the core keyword;

[0033] S530: Determine the dispersion deviation of the core keyword according to the dispersion of the total samples and the frequency deviation of the core keyword.

[0034] Optionally, in S600, determining a recommended ranking of each core keyword based on the dispersion deviation and the deviation value of the core keyword, performing subject matching according to the recommended ranking of each core keyword, and generating recommended teaching subjects based on the matched subjects include:

[0035] S610: After normalizing the dispersion deviation of each core keyword, a weight of each core keyword is obtained, and the deviation value of each core keyword is multiplied by the corresponding weight to obtain a comprehensive score for each core keyword;

[0036] S620, arranging all core keywords in descending order according to the comprehensive scores to obtain a recommended ranking of the core keywords;

[0037] S630, comparing the recommended ranking of the core keywords with the subject keyword matrix in the teaching database, finding the subject keyword with the highest matching degree for each core keyword, and determining the corresponding subject based on the subject keyword;

[0038] S640, generating recommended teaching subjects based on the matched subjects.

[0039] On the other hand, an embodiment of the present invention provides a big data processing system based on industry-education integration, including:

[0040] at least one processor;

[0041] at least one memory for storing at least one program;

[0042] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0043] The beneficial effects of the present invention are as follows: the present invention discloses a big data processing method and system based on the integration of industry and education. The present invention first obtains the target position keyword phrases based on the industry practice database and the teaching database, and then obtains the core phrases through a series of operations such as clustering, and determines the fluctuation characteristic value, deviation value, discreteness deviation, etc. of the core keywords. Finally, based on these data, the core keyword recommendation ranking is determined and the recommended teaching subjects are generated. In this way, the actual needs of the industry can be accurately analyzed and closely integrated with the teaching content, overcoming the problem of the lack of systematic integration of the existing technology, realizing the intelligent, precise and efficient talent training, improving the pertinence and effectiveness of talent training, and cultivating compound talents for the entire industry chain who understand technology, have innovation capabilities and market sensitivity for the textile and garment industry. The present invention can effectively integrate big data analysis with the textile and garment teaching process, and improve the fit between teaching and industry needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a flow chart of a big data processing method based on industry-education integration according to an embodiment of the present invention;

[0046] Figure 2 This is a structural diagram of a big data processing system based on industry-education integration according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects disclosed in the present invention, so as to fully understand the purpose, scheme and effect disclosed in the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict.

[0048] refer to Figure 1 ,like Figure 1 A method for processing big data based on industry-education integration provided by an embodiment of the present invention is shown, and the method includes the following steps:

[0049] S100, obtaining keyword phrases corresponding to M target positions at the latest N sampling times, to obtain M×N keyword phrases; the keyword phrases contain multiple keywords; M and N are both integers greater than 1;

[0050] In the embodiment provided by the present invention, an industry practice database and a teaching database are first obtained, and the industry practice database includes multiple target positions and corresponding keyword phrases, and N groups of keyword phrases are sorted by sampling time; it should be noted that the keyword phrases are key text information used to characterize the positions; the keywords in the keyword phrases are extracted based on the text information in the practical operation information of the positions; the practical operation information is text information used to guide industry employees to perform operations according to the prescribed operating procedures; the practical operation information is provided and authorized for use by enterprises in school-enterprise cooperation, and the latest industry technologies can be understood through the practical operation information, so that by extracting keyword phrases, keyword phrases related to the needs are obtained to guide subsequent subject recommendations.

[0051] S200, clustering the keyword phrases corresponding to the M target positions according to the sampling time to obtain N groups of core phrases; the core phrases contain multiple core keywords;

[0052] Specifically, all keywords in the keyword phrases corresponding to M target positions at a sampling time are taken as a set and clustered to obtain multiple core keywords, which are then formed into a group of core phrases; the core phrases at N sampling times form N groups of core phrases.

[0053] S300, determining a fluctuation characteristic value of each core keyword based on a change trend and a distribution dispersion of the core keywords in the N groups of core phrases;

[0054] In this step, we first need to count the number of times each core keyword appears at different sampling time points and calculate its frequency. This helps us understand the changing trends of keywords over time. By analyzing the frequency changes of core keywords at different sampling time points, we can analyze the changing trends of keywords. This helps us identify which keywords have become more important or less important during a specific time period. By combining the evaluation of the frequency changes and distribution dispersion of keywords at different sampling time points, we can accurately reflect the dynamics of job skills and provide data support for subsequent subject recommendations.

[0055] S400, determining a deviation value of each core keyword based on a difference between each core keyword at each adjacent sampling time and a difference between the fluctuation characteristic values;

[0056] Specifically, the difference between keyword sets at adjacent sampling times is calculated, for example, by calculating the cosine distance of word vectors to measure the degree of change of keywords. At the same time, the difference in the fluctuation characteristic values of adjacent sampling times is compared, and these two difference values are comprehensively calculated, such as weighted summation, to obtain the deviation value of each core keyword at each sampling time. Through such a calculation method, the actual changes and stability of job requirements at different times can be reflected more comprehensively and accurately, providing a solid data foundation for the subsequent precise adjustment of teaching subject content according to job requirements. Target phrases with small deviation values, that is, relatively stable, are identified. The knowledge areas and skills corresponding to these target phrases are strengthened as key content in the subject training process, so that students can have a solid grasp of the key knowledge and skills required for long-term positions; and for keywords with large deviation values, their changing trends are analyzed, and the teaching content is adjusted in a timely manner, so that subject training keeps pace with the dynamic changes in the actual needs of the industry.

[0057] S500: Sampling the total sample consisting of all keywords for all target positions multiple times, and determining the dispersion deviation of each core keyword based on the dispersion of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling;

[0058] S600: Determine the recommended ranking of each core keyword based on the discreteness deviation and the deviation value of the core keyword, perform subject matching based on the recommended ranking of each core keyword, and generate recommended teaching subjects based on the matched subjects.

[0059] In the embodiments provided by this invention, by integrating industry and teaching data, a multidimensional data model is constructed to enable multidimensional data correlation analysis, accurately mapping the gap between industry needs and student capabilities, and providing a scientific basis for optimizing teaching content. Dynamic data updates and continuous model optimization ensure that teaching plans respond to market changes in real time, improving the alignment of talent development with industry needs.

[0060] As an improvement to the above embodiment, in S100, the keyword phrases corresponding to the M target positions at the latest N sampling times are obtained to obtain M×N keyword phrases, including:

[0061] S110, obtaining an industry practice database and a teaching database, wherein the industry practice database includes multiple target positions and corresponding practical operation information;

[0062] S120, extracting keywords from the practical operation information of the target position in order of sampling time to obtain N groups of position keywords;

[0063] S130 , sorting N groups of job keywords according to the time dimension to obtain a job keyword matrix.

[0064] For example, in the job keyword matrix, each row corresponds to the keywords of a target job at different sampling times, and each column corresponds to the keywords of different target jobs at the same sampling time. By organizing and arranging the keywords in this way, it is possible to clearly present the changes in the keywords of each job over time, as well as the differences in keywords of different jobs at the same time, providing a more regular data basis for subsequent cluster analysis. At the same time, after obtaining the teaching database, the knowledge content related to each target job can be analyzed, and the key knowledge points can be extracted and associated with the keyword phrases in the industrial practice database to further explore the intrinsic connection between industry needs and teaching content, so as to better achieve the integration of industry and education. For example, if the keyword phrases for the clothing design job in the industrial practice database at a certain sampling time involve "retro style" and "new fabric application", then the key knowledge points related to retro style design theory and new fabric knowledge can be found in the teaching database, and the coverage of the teaching content for these industry needs can be analyzed so as to adjust the teaching plan in a targeted manner to ensure that the knowledge learned by students is closely integrated with the actual needs of the industry.

[0065] As an improvement to the above embodiment, in S200, the keyword phrases corresponding to the M target positions are clustered according to the sampling time to obtain N groups of core phrases, including:

[0066] S210, performing frequency statistics on the keyword phrases at each sampling time to filter out high-frequency keywords;

[0067] In this step, we count the frequency of each keyword in the keyword phrase at each sampling time. We then set a frequency threshold and select keywords with frequencies above this threshold as high-frequency keywords. This highlights the key information that appears frequently at each sampling time point, providing a more representative vocabulary foundation for the subsequent formation of core phrases.

[0068] S220, calculating the similarity between high-frequency keywords and constructing a similarity matrix; clustering the high-frequency keywords using an agglomerative hierarchical clustering algorithm based on the similarity matrix to obtain multiple high-frequency keyword sets;

[0069] In this step, high-frequency keywords are clustered using a similarity matrix to obtain multiple sets of high-frequency keywords. Specifically, each high-frequency keyword is treated as a separate class. The distance between classes is calculated using the average inter-class distance, and the two classes with the closest distance are merged to form a new class. This process continues until the inter-class distance is greater than a preset distance threshold, and the resulting class is used as the clustering result.

[0070] S230 , grouping the high-frequency keywords in each high-frequency keyword set into a group of high-frequency phrases, and performing weight distribution on the N groups of high-frequency phrases to obtain N groups of core phrases.

[0071] Specifically, we assign weights to each high-frequency keyword based on its importance in practical industry information and the opinions of industry experts. This weighted high-frequency phrase is then derived and used as the core phrase. This weighting helps highlight the importance of different keywords within the core phrase, providing a more accurate basis for subsequent subject recommendations.

[0072] In this step, the weights of high-frequency keywords in high-frequency phrases are adjusted based on the development trends and key areas of focus of the position. For example, if a position has recently increased its attention to technologies related to quick-drying fabrics, the weight of high-frequency keywords related to quick-drying fabrics in the high-frequency phrases will be increased accordingly. Alternatively, based on expert opinions, the high-frequency keywords in the high-frequency phrases can be manually reviewed, and some high-frequency keywords that are unclear or do not meet actual needs can be corrected. Ensure that the core phrases can accurately reflect the key information required for the position, so that the core phrases are always closely aligned with the actual needs of the position, and provide a more accurate basis for subsequent adjustments to teaching content and subject recommendations.

[0073] As an improvement to the above embodiment, in S300, determining the fluctuation characteristic value of each core keyword based on the change trend and distribution dispersion of the core keywords in the N groups of core phrases includes:

[0074] S310, calculating the frequency of occurrence of each core keyword at each sampling time point, fitting a curve showing the change of frequency over time, and using the slope of the curve as the change trend of the core keyword;

[0075] In this step, we calculate the frequency of each core keyword at each sampling time point, plot the frequency over time, and observe whether the curve rises, falls, or stabilizes to understand the changes in the popularity of core keywords over time. For example, if the frequency of a core keyword shows a continuous upward trend, it indicates that the demand for the position represented by this keyword is growing; if the frequency of occurrence decreases, it indicates that the relevant demand may be decreasing.

[0076] S320, obtaining the distribution dispersion of the core keyword based on the variance of the frequency of occurrence of the core keyword at each sampling time point;

[0077] In this step, we use statistical methods to calculate variance to measure the dispersion of the core keyword's frequency of occurrence at each sampling time point. A greater degree of dispersion indicates greater variation in the frequency of core keywords at different times, reflecting significant fluctuations in job demand. Conversely, a smaller degree of dispersion indicates relatively stable job demand.

[0078] S330 , taking the product of the slope and variance of the core keyword at each sampling time point as the characteristic value of the sampling time point, and accumulating the characteristic values of each sampling time to obtain the fluctuation characteristic value of the core keyword.

[0079] In this step, the fluctuation characteristic value of each core keyword is determined based on its changing trend and distribution dispersion. For example, when the core keyword's frequency trend is obvious and its distribution dispersion is large, a higher fluctuation characteristic value is assigned to indicate that the job demand at that sampling time point is changing more dramatically. If the core keyword's frequency change is gentle and its distribution dispersion is small, a lower fluctuation characteristic value is assigned to indicate that the job demand is relatively stable. In this way, the changing trend and distribution dispersion of the core keywords are quantified as fluctuation characteristic values, providing accurate and intuitive data support for subsequent subject recommendations based on job demand.

[0080] As an improvement to the above embodiment, in S400, determining the deviation value of each core keyword based on the difference between each core keyword at each adjacent sampling time and the difference between the fluctuation characteristic values includes:

[0081] S410: For each core keyword, the core keyword is used as the target keyword, the core phrase containing the target keyword is used as the target phrase, the target phrases at adjacent sampling times are converted into word vector representations, and the cosine distance between the two word vectors is calculated using the cosine similarity formula as the difference between the target keywords;

[0082] For example, suppose the target phrases at adjacent sampling times are A and B, and sets A and B contain target keywords; each core keyword in sets A and B is converted into a vector form through the word vector model, and then the cosine distance between them is calculated according to the cosine similarity formula: cos(A,B)=(A·B) / (||A||||B||). The larger the distance value, the greater the difference between the two groups of target phrases, and the greater the change in the knowledge and skills required for the positions related to the target keywords.

[0083] S420, calculating the difference between the fluctuation characteristic values of the target keyword at adjacent sampling times, and taking the absolute value thereof as the difference in the fluctuation characteristic value of the target keyword;

[0084] For example, if the fluctuation characteristic value of the previous sampling time is X1 and the fluctuation characteristic value of the next sampling time is X2, then the difference in the fluctuation characteristic values is |X1-X2|, which reflects the change in the degree of fluctuation in job demand between adjacent sampling times.

[0085] S430 , according to a preset weight coefficient, weighted sums are taken of the differences of the target keyword and the differences of the fluctuation characteristic values at each sampling time, and then accumulated to obtain a deviation value of the target keyword.

[0086] For example, if the weight coefficient of the target keyword difference is set to α, the weight coefficient of the fluctuation characteristic value difference is set to β, and α+β=1, then the deviation value of the target keyword at a certain sampling time = α×difference of the target keyword + β×difference of the fluctuation characteristic value. The deviation value of the target keyword is obtained by accumulating the deviation values of each sampling time.

[0087] The greater the changes in the knowledge and skills required for a position, and the greater the fluctuation in job demand, the greater the changes in the technology corresponding to the position, and the need to update the corresponding disciplines in a timely manner. The corresponding core keywords can be used as the focus for discipline matching; conversely, the smaller the changes in the knowledge and skills required for a position, and the smaller the fluctuation in job demand, the smaller the changes in the technology corresponding to the position, and the position can be regarded as conventional technology, reducing the emphasis on discipline matching; by reasonably setting the weight coefficient, the impact of keyword changes and demand fluctuations on the degree of deviation from job demand can be comprehensively considered, providing a more scientific basis for the subsequent precise adjustment of teaching content.

[0088] As an improvement to the above embodiment, in S500, the total sample consisting of all keywords for all target positions is sampled multiple times, and based on the discreteness of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling, the discreteness deviation of each core keyword is determined, including:

[0089] S510, randomly dividing the total sample into several sub-samples, calculating the dispersion degree of the keywords in each sub-sample respectively, and taking the average of the dispersion degrees as the dispersion degree of the total sample;

[0090] S520, counting the frequency of occurrence of each core keyword in each sampling, and using the standard deviation of the frequency of occurrence in each sampling as the frequency deviation of the core keyword;

[0091] S530: Determine the dispersion deviation of the core keyword according to the dispersion of the total samples and the frequency deviation of the core keyword.

[0092] It should be noted that the core keyword dispersion deviation is calculated by taking the difference between the total sample dispersion and the core keyword frequency deviation, and then finding the absolute value. The core keyword dispersion reflects the fluctuation of job requirements over time. By comparing the core keyword frequency deviation with the total keyword dispersion, we can accurately quantify the relative magnitude of changes in job requirements corresponding to the core keywords, thereby providing strong support for adjusting teaching content. This method can not only effectively capture subtle changes in job requirements, but also accurately grasp their changing trends, thereby ensuring a high degree of alignment between teaching content and market demand, and improving the targeted and effective nature of talent development.

[0093] The calculation formula for the dispersion of core keywords is: , where fi is the frequency deviation of core keyword k in the i-th sampling, F is the average frequency of each keyword across all samples, and n is the number of samples. This formula can quantify the fluctuation of each core keyword at different sampling times, further refining the capture of dynamic changes in job requirements and providing more accurate data support for teaching content optimization.

[0094] As an improvement to the above embodiment, in S600, the process of determining the recommended ranking of each core keyword based on the dispersion deviation and the deviation value of the core keyword, performing subject matching according to the recommended ranking of each core keyword, and generating recommended teaching subjects based on the matched subjects includes:

[0095] S610: After normalizing the dispersion deviation of each core keyword, a weight of each core keyword is obtained, and the deviation value of each core keyword is multiplied by the corresponding weight to obtain a comprehensive score for each core keyword;

[0096] The deviation value of the core keyword reflects the actual changes and stability of job requirements, and the discrete deviation of the core keyword reflects the relative intensity of changes in job requirements. After normalizing the discrete deviation that represents relative changes, the deviation value that represents absolute changes is weighted to obtain the comprehensive score of each core keyword. This comprehensive score can comprehensively reflect the importance of the core keyword based on changes in job requirements.

[0097] S620, arranging all core keywords in descending order according to the comprehensive scores to obtain a recommended ranking of the core keywords;

[0098] The higher the score of a core keyword, the more important and active its change in job requirements, and it should be given a higher priority in subject matching.

[0099] S630, comparing the recommended ranking of the core keywords with the subject keywords in the teaching database, finding the subject keyword with the highest matching degree for each core keyword, and determining the corresponding subject based on the subject keyword;

[0100] Compare the recommended ranking of core keywords with the subject keywords in the teaching database. For each core keyword, find the subject keyword with the highest matching degree to determine the corresponding subject. During this comparison process, text similarity algorithms such as the edit distance algorithm and TF-IDF algorithm can be used to accurately measure the similarity between core keywords and subject keywords.

[0101] S640, generating recommended teaching subjects based on the matched subjects.

[0102] Recommended teaching disciplines are generated based on matching disciplines and combined with practical teaching conditions, such as curriculum and faculty. Disciplines with a high degree of matching and meeting practical teaching requirements are prioritized for inclusion in the teaching system. At the same time, the teaching content and course schedule of these recommended disciplines are adjusted accordingly to ensure that the teaching content closely aligns with the actual job requirements and cultivates professionals that better meet the needs of industrial development.

[0103] refer to Figure 2 , an embodiment of the present invention further provides a big data processing system based on industry-education integration, including:

[0104] at least one processor;

[0105] at least one memory for storing at least one program;

[0106] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0107] The contents of the above method embodiments are all applicable to this embodiment. The functions specifically implemented by this embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments, which will not be repeated here.

[0108] Although the description of the present disclosure has been quite detailed and particularly describes several embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be considered to provide a broad possible interpretation of these claims by reference to the appended claims in view of the prior art, thereby effectively covering the intended scope of the present disclosure. In addition, the above description of the present disclosure is based on the embodiments foreseen by the inventors, which is intended to provide a useful description, and those non-substantial changes to the present disclosure that have not yet been foreseen may still represent equivalent changes to the present disclosure.

Claims

1. A big data processing method based on industry-education integration, characterized in that: The method comprises the following steps: S100, obtaining keyword phrases corresponding to M target positions at the latest N sampling times, to obtain M×N keyword phrases; the keyword phrases contain multiple keywords; M and N are both integers greater than 1; S200, clustering the keyword phrases corresponding to the M target positions according to the sampling time to obtain N groups of core phrases; the core phrases contain multiple core keywords; S300, determining a fluctuation characteristic value of each core keyword based on a change trend and a distribution dispersion of the core keywords in the N groups of core phrases; S400, determining a deviation value of each core keyword based on a difference between each core keyword at each adjacent sampling time and a difference between the fluctuation characteristic values; S500: Sampling the total sample consisting of all keywords for all target positions multiple times, and determining the dispersion deviation of each core keyword based on the dispersion of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling; S600: Determine the recommended ranking of each core keyword based on the discreteness deviation and the deviation value of the core keyword, perform subject matching based on the recommended ranking of each core keyword, and generate recommended teaching subjects based on the matched subjects.

2. A big data processing method based on industry-education integration according to claim 1, characterized in that: In S100, the keyword phrases corresponding to the M target positions at the latest N sampling times are obtained to obtain M×N keyword phrases, including: S110, obtaining an industry practice database and a teaching database, wherein the industry practice database includes multiple target positions and corresponding practical operation information; S120, extracting keywords from the practical operation information of the target position in order of sampling time to obtain N groups of position keywords; S130 , sorting N groups of job keywords according to the time dimension to obtain a job keyword matrix.

3. The big data processing method based on industry-education integration according to claim 2 is characterized in that: In S200, the keyword phrases corresponding to the M target positions are clustered according to the sampling time to obtain N groups of core phrases, including: S210, performing frequency statistics on the keyword phrases at each sampling time to filter out high-frequency keywords; S220, calculating the similarity between high-frequency keywords and constructing a similarity matrix; clustering the high-frequency keywords using an agglomerative hierarchical clustering algorithm based on the similarity matrix to obtain multiple high-frequency keyword sets; S230 , grouping the high-frequency keywords in each high-frequency keyword set into a group of high-frequency phrases, and performing weight distribution on the N groups of high-frequency phrases to obtain N groups of core phrases.

4. The big data processing method based on industry-education integration according to claim 1 is characterized in that: In S300, the step of determining the fluctuation characteristic value of each core keyword based on the change trend and distribution dispersion of the core keywords in the N core phrase groups includes: S310, calculating the frequency of occurrence of each core keyword at each sampling time point, fitting a curve showing the change of frequency over time, and using the slope of the curve as the change trend of the core keyword; S320, obtaining the distribution dispersion of the core keyword according to the variance of the frequency of occurrence of the core keyword at each sampling time point; S330 , taking the product of the slope and variance of the core keyword at each sampling time point as the characteristic value of the sampling time point, and accumulating the characteristic values of each sampling time to obtain the fluctuation characteristic value of the core keyword.

5. The big data processing method based on industry-education integration according to claim 1 is characterized in that: In S400, determining the deviation value of each core keyword according to the difference between each core keyword at each adjacent sampling time and the difference between the fluctuation characteristic values includes: S410: For each core keyword, use the core keyword as the target keyword, use the core phrase containing the target keyword as the target phrase, convert the target phrases at adjacent sampling times into word vector representations, and use the cosine similarity formula to calculate the cosine distance between the two word vectors as the difference between the target keywords; S420, calculating the difference between the fluctuation characteristic values of the target keyword at adjacent sampling times, and taking the absolute value thereof as the difference in the fluctuation characteristic value of the target keyword; S430 , according to a preset weight coefficient, weighted sums are taken of the differences of the target keyword and the differences of the fluctuation characteristic values at each sampling time, and then accumulated to obtain a deviation value of the target keyword.

6. The big data processing method based on industry-education integration according to claim 1 is characterized in that: In S500, the total sample consisting of all keywords for all target positions is sampled multiple times, and based on the discreteness of the keywords obtained from the multiple samplings and the frequency deviation of the core keywords in each sampling, the discreteness deviation of each core keyword is determined, including: S510, randomly dividing the total sample into several sub-samples, calculating the dispersion degree of the keywords in each sub-sample respectively, and taking the average of the dispersion degrees as the dispersion degree of the total sample; S520, counting the frequency of occurrence of each core keyword in each sampling, and using the standard deviation of the frequency of occurrence in each sampling as the frequency deviation of the core keyword; S530: Determine the dispersion deviation of the core keyword according to the dispersion of the total samples and the frequency deviation of the core keyword.

7. The big data processing method based on industry-education integration according to claim 1 is characterized in that: In S600, the process of determining the recommended ranking of each core keyword based on the dispersion deviation and the deviation value of the core keyword, performing subject matching according to the recommended ranking of each core keyword, and generating recommended teaching subjects based on the matched subjects includes: S610: After normalizing the dispersion deviation of each core keyword, a weight of each core keyword is obtained, and the deviation value of each core keyword is multiplied by the corresponding weight to obtain a comprehensive score for each core keyword; S620, arranging all core keywords in descending order according to the comprehensive scores to obtain a recommended ranking of the core keywords; S630, comparing the recommended ranking of the core keywords with the subject keyword matrix in the teaching database, finding the subject keyword with the highest matching degree for each core keyword, and determining the corresponding subject based on the subject keyword; S640, generating recommended teaching subjects based on the matched subjects.

8. A big data processing system based on industry-education integration, characterized by: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

Images