Veterinary epidemiological survey and analysis system based on big data technology

Abstract

The present invention relates to the field of survey and analysis technology, and more specifically, to a veterinary epidemiological survey and analysis system based on big data technology. The system comprises a data similarity unit, a criticality unit, an important association unit, and an analysis and determination unit. The data similarity unit calculates the similarity between a word and an animal, the similarity between a word and a disease, and the word frequency based on the norm of an animal vector, the norm of a disease vector, and the norm of a collected word vector. The criticality unit of the present invention calculates the inverse document frequency of a word based on the number of documents and the number of documents containing the word, and then uses the inverse document frequency and the word frequency to calculate the criticality value of the word in the document. When the criticality value is greater than a set criticality value, the word is named a keyword. The criticality can filter out frequently appearing useless words, and prevent the frequent appearance of useless words from being mistakenly regarded as key information, thereby highlighting truly important information and improving the efficiency of extracting important information.

Description

Technical Field

[0001] The present invention relates to the field of investigation and analysis technology, and in particular to a veterinary epidemiological investigation and analysis system based on big data technology. Background Art

[0002] The veterinary epidemiology survey and analysis system based on big data technology refers to the use of big data technologies (such as data mining, machine learning, and artificial intelligence) to conduct efficient, comprehensive, and in-depth analysis of veterinary epidemiological data, thereby accurately identifying, monitoring, and effectively managing animal diseases, improving the management level of animal diseases, achieving early warning and rapid response, ensuring public health safety, supporting scientific decision-making, and promoting academic research. It is of great significance to modern veterinary medicine and public health, and provides powerful tools and methods for animal health protection;

[0003] Since animals may become sick during the process of being raised on farms, in order to prevent the sick animals from causing harm to other animals or humans, data related to sick animals are investigated through multiple channels. When it is necessary to extract key information from the investigated data related to sick animals, a large number of useless words will frequently appear, causing the frequently appearing useless words to be mistakenly regarded as key information. Important information will be buried in a large amount of irrelevant information, and the truly important information cannot be highlighted, resulting in reduced efficiency in extracting important information. At the same time, since important information will be buried in a large amount of irrelevant information, irrelevant or low-importance statements will not be filtered out in time. Processing irrelevant statements will consume more computing resources, increase computing time, and thus reduce processing efficiency. Therefore, we provide a veterinary epidemiological survey and analysis system based on big data technology. Summary of the Invention

[0004] The purpose of the present invention is to provide a veterinary epidemiological survey and analysis system based on big data technology to solve the problems raised in the above background technology.

[0005] To achieve the above-mentioned object, the present invention provides a veterinary epidemiological investigation and analysis system based on big data technology, comprising a data similarity unit, a criticality unit, an important association unit, and an analysis and determination unit;

[0006] The data similarity unit investigates data related to diseased animals through multiple channels and obtains historical data. The data related to diseased animals are divided into multiple document data according to the investigated data. Then, words are collected from the documents. The norm of the animal vector, the norm of the disease vector, and the norm of the collected word vector are calculated based on the collected words and historical data. The similarity between the word and the animal, the similarity between the word and the disease, and the word frequency are calculated.

[0007] The key degree unit is used to receive the document and word frequency in the data similarity unit, and calculate the key degree value of the word in the document using the inverse word frequency. When the key degree value is greater than the set key degree value, it indicates that the word is key in the document and the word is named as a keyword;

[0008] The important association unit is used to receive the keywords and word inverse document frequencies in the documents and the criticality unit in the data similarity unit, record the number of keywords, the number of characters, and the position of sentences in the document, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by using the analyzed importance of the sentences and the average inverse document frequency value of the sentences, and analyze the correlation of adjacent sentences;

[0009] When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the analyzed correlation of adjacent sentences is within the set stability range, the adjacent sentences are considered to have strong correlation;

[0010] When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. If the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation. The corresponding sentence of the analyzed sentence importance is input into the Transformer model, and the key feature information of the sentence is output.

[0011] The analysis and determination unit is used to receive key feature information of adjacent sentences or sentences in important associated units, integrate the key feature information of adjacent sentences or sentences into key documents, perform cluster analysis based on the key documents and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis.

[0012] As a further improvement of the present technical solution, the data similarity unit includes a data calculation module and a similarity module;

[0013] The data calculation module investigates data related to diseased animals through multiple channels and obtains historical data. It then divides the investigated data related to diseased animals into multiple document data and records the number of documents. It then collects words from the documents, inputs the animals, diseases and collected words into a Transformer model in big data technology, and the Transformer model outputs corresponding animal vectors, disease vectors and collected word vectors. The animal vectors, disease vectors and collected word vectors are used to calculate the norm of the animal vector, the norm of the disease vector and the norm of the collected word vector.

[0014] As a further improvement of the present technical solution, the similarity module is used to receive the animal vector, the disease vector, the collected word vector and the norm of the animal vector, the norm of the disease vector, the norm of the collected word vector, and the document in the data calculation module, and calculate the similarity between the word and the animal;

[0015] When the similarity between words and animals is close , indicating that the word and animal are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document.

[0016] When the similarity between words and animals is close When , it indicates that there is no semantic correlation between the word and the animal. Then the similarity between the word and the disease is calculated using the disease vector, the collected word vector, the norm of the disease vector, and the norm of the collected word vector;

[0017] When the similarity between words and diseases is close , indicating that the word and disease are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document.

[0018] As a further improvement of this technical solution, the criticality unit is used to receive the number of documents in the data calculation module, the number of documents containing words in the document and similarity module, and the word frequency, and calculate the word inverse document frequency according to the number of documents and the number of documents containing words. , and then use the inverse document frequency and word frequency to calculate the key value of the word in the document When the criticality value is greater than the set criticality value, it means that the word is key in the document and the word is named as a keyword.

[0019] As a further improvement of the present technical solution, the important association unit includes an importance module and a relevance module;

[0020] The importance module is used to receive keywords in the document and criticality unit in the data calculation module, record the number of keywords, number of characters, and position of the sentence in the document, input the number of keywords, number of characters, and position into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight, and uses the corresponding weight coefficient, position weight, number of keywords, and number of characters to analyze the importance of the sentence.

[0021] As a further improvement of this technical solution, the importance module analyzes the importance of sentences according to the following principles:

[0022] Collect the corresponding weight coefficients 、 , position weight , number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include 、 ,..., 、 , the specific algorithm formula is:

[0023] .

[0024] As a further improvement of this technical solution, the relevance module is used to receive the importance of the analysis statement in the importance module, the number of keywords and the inverse document frequency of words in the key degree unit, and obtain the importance of the analysis statement from the importance of the analysis statement. The importance of analyzing statements , using the importance of analyzing sentences The importance of analyzing statements Calculating the change amount of adjacent sentences and the normalized variables of the high importance of adjacent sentences, analyzing the correlation of adjacent sentences based on the change amount of adjacent sentences and the normalized variables of the high importance of adjacent sentences, and obtaining the analyzed correlation of adjacent sentences;

[0025] When the change of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation, and the importance of the analyzed sentences is The importance of analyzing statements The corresponding adjacent statements and statements Enter into the analysis and determination unit;

[0026] When the change of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. When the correlation of the analyzed adjacent sentences exceeds the set stability range, it is considered that the adjacent sentences have weak correlation. The importance of analyzing statements After comparison, the sentences with lower importance are input into the Transformer model. The Transformer model outputs the key feature information of the sentences, and the sentences with higher importance and the key feature information of the sentences are input into the analysis and determination unit.

[0027] As a further improvement of the present technical solution, the analysis and determination unit is used to receive the corresponding adjacent statements in the relevance module. and statements Or the key feature information of the corresponding higher sentences and sentences, all the corresponding adjacent sentences and statements Or the corresponding relatively high sentences and the key feature information of the sentences are input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. The extracted key data and key documents are used for cluster analysis, and then the animal diseases are identified based on the results of the cluster analysis.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. In the veterinary epidemiological survey and analysis system based on big data technology, the criticality unit calculates the inverse document frequency of a word according to the number of documents and the number of documents containing the word, and then uses the inverse document frequency and the word frequency to calculate the criticality value of the word in the document. When the criticality value is greater than the set criticality value, it means that the word is key in the document, and the word is named as a keyword. The criticality can filter out frequently appearing useless words and avoid frequently appearing useless words being mistakenly regarded as key information, highlighting the truly important information and improving the efficiency of extracting important information.

[0030] 2. In this veterinary epidemiological survey and analysis system based on big data technology, the importance module records the number of keywords and characters in each sentence in the document, as well as the position of the sentence in the document. The number of keywords, number of characters and position are input into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight. The importance of the sentence is analyzed using the corresponding weight coefficient, position weight, number of keywords and number of characters. Through sentence importance analysis, those irrelevant or low-importance sentences can be effectively filtered out, the number of processed sentences can be reduced, the time for data processing and analysis can be significantly shortened, and processing efficiency can be improved.

[0031] 3. In the veterinary epidemiological survey and analysis system based on big data technology, the correlation module calculates the average inverse frequency value of the sentence according to the number of keywords and the inverse document frequency of the words, and then takes the sentence inverse frequency value from the average inverse frequency value of the sentence, calculates the change in the inverse frequency value of the adjacent sentence according to the sentence inverse frequency value, and normalizes the importance of the analyzed sentence with the high importance of the adjacent sentence. The correlation of the adjacent sentences is analyzed according to the change in the inverse frequency value of the adjacent sentence and the normalized variable of the high importance of the adjacent sentence, and the correlation of the analyzed adjacent sentences is obtained. The sentence inverse frequency can be used to identify those sentences that appear less frequently in the entire document collection but are important, so as to avoid the mistaken exclusion of important sentences that appear less frequently in the document, thereby improving the accuracy of the retrieval of important sentences. At the same time, the correlation of adjacent sentences is analyzed based on the sentence inverse frequency, and key feature information is extracted for sentences with low correlation, thereby reducing the number of low-correlation sentences that need to be processed and reducing the computational complexity. In particular, the processing speed is improved in large-scale text processing and high-concurrency tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A unit block diagram of the present invention;

[0033] Figure 2 It is a block diagram of a module unit of the present invention;

[0034] Figure 3 A block diagram of a data similarity unit of the present invention;

[0035] Figure 4 It is a block diagram of the important related units of the present invention.

[0036] The meaning of each number in the figure is:

[0037] 1. Data similarity unit; 11. Data calculation module; 12. Similarity module; 2. Criticality unit;

[0038] 3. Important association unit; 31. Importance module; 32. Correlation module; 4. Analysis and determination unit. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] The present invention provides a veterinary epidemiological survey and analysis system based on big data technology, please refer to Figure 1, including data similarity unit 1, criticality unit 2, important association unit 3, analysis and determination unit 4;

[0041] Data similarity unit 1 investigates data related to diseased animals through multiple channels and obtains historical data. The investigated data related to diseased animals is divided into multiple document data. Then, words are collected from the documents. Based on the collected words and historical data, the norm of the animal vector, the norm of the disease vector, and the norm of the collected word vector are calculated. The similarity between the word and the animal, the similarity between the word and the disease, and the word frequency are calculated.

[0042] The key degree unit 2 is used to receive the document and word frequency in the data similarity unit 1, calculate the key degree value of the word in the document using the inverse word frequency, and name the word as a keyword when the key degree value is greater than the set key degree value;

[0043] The important association unit 3 is used to receive the keywords and word inverse document frequency in the document in the data similarity unit 1 and the criticality unit 2, record the number of keywords, the number of characters, and the position of the sentences in the document, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by using the analyzed importance of the sentences and the average inverse document frequency value, and analyze the correlation of adjacent sentences;

[0044] When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the analyzed correlation of adjacent sentences is within the set stability range, the adjacent sentences are considered to have strong correlation;

[0045] When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. If the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation. The corresponding sentence of the analyzed sentence importance is input into the Transformer model, and the key feature information of the sentence is output.

[0046] The analysis and determination unit 4 is used to receive the key feature information of adjacent sentences or sentences in the important association unit 3, integrate the key feature information of adjacent sentences or sentences into key documents, and then perform cluster analysis based on the key documents and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis.

[0047] The following is a refinement of the above units, see Figure 2-Figure 4 ;

[0048] The data similarity unit 1 includes a data calculation module 11 and a similarity module 12;

[0049] Since animals may get sick during the breeding process, in order to prevent the sick animals from causing harm to other animals or humans, the data calculation module 11 is used to investigate the data related to the sick animals through multiple channels and obtain historical data. The data related to the sick animals investigated are then divided into multiple document data and the number of documents is recorded. , then collect words from the document, input the animals, diseases and collected words into the Transformer model in big data technology, and the Transformer model outputs the corresponding animal vector , disease vector and the collected word vectors , using animal vectors , disease vector and the collected word vectors Calculate the norm of the animal vector , the norm of the disease vector and the norm of the collected word vectors ,in, Refers to the Animal vector , Refers to the Animal vector , Refers to the Disease vectors , Refers to the Disease vectors , Refers to the The collected word vectors , Refers to the The collected word vectors ,By calculating the norm of the animal vector, the disease vector and the ,related word vector, we can quantify the similarity between them, and ,help identify potential correlations;

[0050] The data related to sick animals investigated include case records from veterinary clinics, environmental data from farms, and physiological index data of animals;

[0051] Historical data includes key words for animals and diseases;

[0052] The similarity module 12 is used to receive the animal vector in the data calculation module 11 , disease vector , collected word vectors and the norm of the animal vector , the norm of the disease vector , the norm of the collected word vectors , documents, using animal vectors , collected word vectors and the norm of the animal vector , the norm of the collected word vectors Calculate the similarity between words and animals , where the value range is ,Through similarity analysis, it is possible to identify that the collected words are highly ,correlated with the specific physiological, pathological or disease characteristics of ,animals, and discover new research directions, which is ,facilitated in improving the accuracy of determining the types of ,animal diseases;

[0053] When the similarity between words and animals near , indicating that the word and the word animal are semantically related, and then record the number of times the word appears in the document. and the total number of words in the document , and the number of documents containing the term , using the number of times a word appears in a document and the total number of words in the document Calculating word frequency , which is convenient for subsequent calculation of word inverse document frequency;

[0054] When the similarity between words and animals near When , it means that there is no semantic correlation between the word and the animal. Then, the disease vector , collected word vectors and the norm of the disease vector , the norm of the collected word vectors Calculate the similarity between words and diseases ,Through similarity analysis, it is possible to identify that the collected words are highly ,correlated with the specific physiological, pathological or disease characteristics of ,animals, and discover new research directions, which is ,facilitated in improving the accuracy of determining the types of ,animal diseases;

[0055] When the similarity between words and diseases near , indicating that the word and the disease are semantically related, and then record the number of times the word appears in the document. and the total number of words in the document , and the number of documents containing the term , using the number of times a word appears in a document and the total number of words in the document Calculating word frequency , which is convenient for subsequent calculation of word inverse document frequency;

[0056] The criticality unit 2 is used to receive the number of documents in the data calculation module 11 , documents and the number of documents containing words in the similarity module 12 , word frequency or word frequency , according to the number of documents and the number of documents containing the term Calculate the inverse document frequency of a term , where if the number of documents containing the word When , it means that there is no word in all documents. When the word frequency is greater than the set inverse document frequency, it means that the word is not common in the document and is critical in the document. , which can identify those words that appear less frequently in the entire document collection but are key words, avoiding the mistaken exclusion of key words that appear less frequently in the document, improving the accuracy of key word retrieval, and then using the word inverse document frequency and word frequency or word frequency Calculate the criticality of words in documents or , when the criticality value Greater than the set criticality value When the word is key in the document, it is named as a keyword. The key degree can be used to filter out useless frequently appearing words and avoid frequently appearing words being mistakenly regarded as key information, highlighting the truly important information and improving the efficiency of extracting important information.

[0057] Suppose we have 100 documents and consider the following two words:

[0058] Term A appears in 10 documents, so :

[0059]

[0060] Term B appears in 2 documents, so :

[0061]

[0062] Word B The value is higher than that of word A. Word B is less common in the document, which means that word B has higher discrimination and information content and can more effectively indicate the main topic of the document containing it.

[0063] The important association unit 3 includes an importance module 31 and a relevance module 32;

[0064] The importance module 31 is used to receive the document in the data calculation module 11 and the keywords in the criticality unit 2, and record the number of keywords in each sentence in the document. and the number of characters , and the position of the statement in the document , the number of keywords , number of characters and location Input into the Transformer model, the Transformer model outputs the corresponding weight coefficient 、 and position weight , using the corresponding weight coefficient 、 , position weight , number of keywords and the number of characters Analyze the importance of the statement. When the importance of the analyzed statement is greater than the set importance value, it means that the importance value of the statement is high. Through statement importance analysis, those insignificant or low-importance statements can be effectively filtered out, reducing the number of processed statements, significantly shortening the time for data processing and analysis, and improving processing efficiency. Commands with high importance values are passed to the relevance module 32;

[0065] The principle of analyzing the importance of statements:

[0066] Collect the corresponding weight coefficients 、 , position weight , number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include 、 ,..., 、 , the specific algorithm formula is:

[0067] ;

[0068] in, 、 There are two weight coefficients used to adjust the number of keywords and position weight Through sentence importance analysis, we can effectively filter out those irrelevant or low-importance sentences, which helps reduce noise and make the truly important information more prominent;

[0069] The relevance module 32 is used to receive the importance of the analysis statement in the importance module 31 , number of keywords and the inverse document frequency of words in criticality unit 2 , according to the number of keywords and term inverse document frequency Calculate the average reverse frequency value of the statement ,in, Refers to the The inverse document frequency of each word, and then the average inverse document frequency value of the sentence Take Statement and statements The frequency value of the statement inverse and , according to the statement reverse frequency value and Calculate the change in the reverse frequency value of adjacent sentences , and then analyze the importance of the sentence Get the importance of the analysis statement The importance of analyzing statements , will analyze the importance of the sentence The importance of analyzing statements Normalize variables with high importance of adjacent sentences and , according to the change in the reverse frequency value of the adjacent sentence Normalized variables with high importance of adjacent sentences 、 Analyze the relevance of adjacent sentences and obtain the relevance of adjacent sentences analyzed Through the reverse frequency of sentences, we can identify those sentences that appear less frequently in the entire document collection but are important, avoid mistakenly excluding important sentences that appear less frequently in the document, and improve the accuracy of important sentence retrieval. At the same time, based on the reverse frequency of sentences, we analyze the relevance of adjacent sentences and extract key feature information from sentences with low relevance, thereby reducing the number of low-relevance sentences that need to be processed and reducing computational complexity. This improves processing speed, especially in large-scale text processing and high-concurrency tasks.

[0070] When the frequency value of the adjacent statement changes in reverse order Within the set difference range, it means that the adjacent sentences are very similar in importance. If the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation. The importance of analyzing statements The corresponding adjacent statements and statements Entered into analysis and determination unit 4;

[0071] When the frequency value of the adjacent statement changes in reverse order If the difference exceeds the set range, it means that the adjacent sentences are not similar in importance. If the correlation of the analyzed adjacent sentences exceeds the set stability range, it is considered that the adjacent sentences have weak correlation. The importance of analyzing statements Compare and analyze the importance of the sentences The sentence with a relatively low correspondence is input into the Transformer model, and the Transformer model outputs the key feature information of the sentence, and the sentence with a relatively high correspondence and the key feature information of the sentence are input into the analysis and determination unit 4;

[0072] The analysis and determination unit 4 is used to receive the corresponding adjacent sentences in the relevance module 32 and statements Or the key feature information of the corresponding higher sentences and sentences, all the corresponding adjacent sentences and statements Or the key feature information of the corresponding higher sentences and sentences is input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. By analyzing the data associated with the key documents, the key factors affecting animal health can be identified, providing accurate data for subsequent research, and cluster analysis is performed using the extracted key data and key documents. Then, based on the results of the cluster analysis, animal diseases are identified. Through cluster analysis of key data and key documents, key variables related to specific diseases can be accurately identified, thereby improving the accuracy of disease identification.

[0073] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. Veterinary epidemiological survey and analysis system based on big data technology, characterized by: It includes data similarity unit (1), criticality unit (2), important association unit (3), and analysis and determination unit (4); The data similarity unit (1) investigates data related to diseased animals through multiple channels and obtains historical data, divides the investigated data related to diseased animals into multiple document data, collects words from the documents, calculates the norm of the animal vector, the norm of the disease vector, and the norm of the collected word vector based on the collected words and historical data, and calculates the similarity between the word and the animal, the similarity between the word and the disease, and the word frequency; The criticality unit (2) is used to receive the documents and word frequencies in the data similarity unit (1), and calculate the criticality value of the word in the document using the inverse word frequency. When the criticality value is greater than the set criticality value, it indicates that the word is critical in the document, and the word is named as a keyword. The important association unit (3) is used to receive the key words and word inverse document frequencies in the document in the data similarity unit (1) and the key degree unit (2), record the number of key words, the number of characters, and the position of sentences in the document, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by using the analyzed importance of the sentences and the average inverse document frequency value of the sentences, and analyze the correlation of adjacent sentences; When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the analyzed correlation of adjacent sentences is within the set stability range, the adjacent sentences are considered to have strong correlation; When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. If the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation. The corresponding sentence of the analyzed sentence importance is input into the Transformer model, and the key feature information of the sentence is output. The analysis and determination unit (4) is used to receive key feature information of adjacent sentences or sentences in the important association unit (3), integrate the key feature information of the adjacent sentences or sentences into a key document, perform cluster analysis based on the key document and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis; The important association unit (3) includes an importance module (31) and a relevance module (32); The importance module (31) is used to receive the keywords in the document and the criticality unit (2) in the data calculation module (11), record the number of keywords, the number of characters, and the position of the sentence in the document, input the number of keywords, the number of characters, and the position into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight, and analyzes the importance of the sentence using the corresponding weight coefficient, position weight, number of keywords, and number of characters; The importance module (31) implements the principle of analyzing the importance of sentences: Collect the corresponding weight coefficients 、 , position weight , number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include 、 ,..., 、 , the specific algorithm formula is: 。 2. The veterinary epidemiological survey and analysis system based on big data technology according to claim 1, characterized in that: The data similarity unit (1) includes a data calculation module (11) and a similarity module (12); The data calculation module (11) investigates data related to diseased animals through multiple channels and obtains historical data, then divides the investigated data related to diseased animals into multiple document data, records the number of documents, and then collects words from the documents, inputs animals, diseases and collected words into the Transformer model in big data technology, and the Transformer model outputs corresponding animal vectors, disease vectors and collected word vectors, and uses the animal vectors, disease vectors and collected word vectors to calculate the norm of the animal vector, the norm of the disease vector and the norm of the collected word vector.

3. The veterinary epidemiological survey and analysis system based on big data technology according to claim 2, characterized in that: The similarity module (12) is used to receive the animal vector, the disease vector, the collected word vector and the norm of the animal vector, the norm of the disease vector, the norm of the collected word vector, and the document in the data calculation module (11), and calculate the similarity between the word and the animal; When the similarity between words and animals is close , indicating that the word and animal are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document. When the similarity between words and animals is close When , it indicates that there is no semantic correlation between the word and the animal. Then the similarity between the word and the disease is calculated using the disease vector, the collected word vector, the norm of the disease vector, and the norm of the collected word vector; When the similarity between words and diseases is close , indicating that the word and disease are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document.

4. The veterinary epidemiological survey and analysis system based on big data technology according to claim 3 is characterized by: The criticality unit (2) is used to receive the number of documents in the data calculation module (11), the number of documents containing words in the document and similarity module (12), and the word frequency, and calculate the word inverse document frequency based on the number of documents and the number of documents containing words , and then use the inverse document frequency and word frequency to calculate the key value of the word in the document When the criticality value is greater than the set criticality value, it means that the word is key in the document and the word is named as a keyword.

5. The veterinary epidemiological survey and analysis system based on big data technology according to claim 1, characterized in that: The relevance module (32) is used to receive the importance of the sentence analyzed in the importance module (31), the number of keywords and the word inverse document frequency in the key degree unit (2), and calculate the average inverse document frequency value of the sentence based on the number of keywords and the word inverse document frequency. ,in, Refers to the The inverse document frequency of each word, and then take the sentence from the average inverse document frequency value and statements The frequency value of the statement inverse and , calculate the change in the reverse frequency value of adjacent sentences based on the reverse frequency value of the sentence , and then obtain the importance of the analysis statement from the importance of the analysis statement The importance of analyzing statements , will analyze the importance of the sentence The importance of analyzing statements Normalize variables with high importance of adjacent sentences and , according to the change in the reverse frequency value of the adjacent sentence Normalized variables with high importance of adjacent sentences 、 Analyze the relevance of adjacent sentences and obtain the relevance of adjacent sentences analyzed ; When the change of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation, and the importance of the analyzed sentences is The importance of analyzing statements The corresponding adjacent statements and statements Entered into the analysis and determination unit (4); When the change of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. When the correlation of the analyzed adjacent sentences exceeds the set stability range, it is considered that the adjacent sentences have weak correlation. The importance of analyzing statements Comparison is made, and sentences with lower importance are input into the Transformer model. The Transformer model outputs key feature information of the sentences, and sentences with higher importance and key feature information of the sentences are input into the analysis and determination unit (4).

6. The veterinary epidemiological survey and analysis system based on big data technology according to claim 1, characterized in that: The analysis and determination unit (4) is used to receive the corresponding adjacent sentences in the relevance module (32) and statements Or the key feature information of the corresponding higher sentences and sentences, all the corresponding adjacent sentences and statements Or the corresponding relatively high sentences and the key feature information of the sentences are input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. The extracted key data and key documents are used for cluster analysis, and then the animal diseases are identified based on the results of the cluster analysis.

Images