Veterinary epidemiological investigation and analysis system based on big data technology

Abstract

The invention relates to the technical field of investigation and analysis, in particular to a veterinary epidemiological investigation and analysis system based on a big data technology. The system comprises a data similarity unit, a key degree unit, an important association unit and an analysis determination unit, wherein the data similarity unit calculates the similarity between words and animals, the similarity between words and diseases and the word frequency according to the norm of animal vectors, the norm of disease vectors and the norm of collected word vectors. The key degree unit calculates the inverse document frequency of the words according to the number of the documents and the number of the documents containing the words, then calculates the key degree values of the words in the documents by using the inverse document frequency and the word frequency, and names the words as keywords when the key degree values are greater than the set key degree values. Through the key degree, frequently appearing useless words can be filtered out, the useless words are prevented from frequently appearing and being mistakenly regarded as key information, really important information is highlighted, and the extraction efficiency of the important information is improved.

Description

Technical Field

[0001] The present invention relates to the field of investigation and analysis technology, and in particular to a veterinary epidemiology investigation and analysis system based on big data technology. Background Art

[0002] The veterinary epidemiology survey and analysis system based on big data technology refers to the use of big data technology (such as data mining, machine learning, artificial intelligence) to conduct efficient, comprehensive and in-depth analysis of veterinary epidemiological data, so as to accurately identify, monitor and effectively manage animal diseases, improve the management level of animal diseases, achieve early warning and rapid response, ensure public health safety, support scientific decision-making, and promote academic research. It is of great significance to the fields of modern veterinary medicine and public health, and provides powerful tools and methods for animal health protection; Since animals may get sick during the process of raising them in farms, in order to prevent the sick animals from causing harm to other animals or humans, data related to sick animals are investigated through multiple channels. When it is necessary to extract key information from the investigated data related to sick animals, a large number of useless words will frequently appear, causing the frequently appearing useless words to be mistakenly regarded as key information. Important information will be buried in a large amount of irrelevant information, and the truly important information cannot be highlighted, resulting in reduced efficiency in extracting important information. At the same time, because important information will be buried in a large amount of irrelevant information, irrelevant or low-importance sentences are not filtered out in time, processing irrelevant sentences will consume more computing resources, increase computing time, and thus reduce processing efficiency. Therefore, we provide a veterinary epidemiology survey and analysis system based on big data technology. Summary of the invention

[0003] The purpose of the present invention is to provide a veterinary epidemiological survey and analysis system based on big data technology to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above-mentioned object, the present invention provides a veterinary epidemiological investigation and analysis system based on big data technology, including a data similarity unit, a criticality unit, an important association unit, and an analysis and determination unit; The data similarity unit investigates data related to diseased animals through multiple channels and obtains historical data, divides the investigated data related to diseased animals into multiple document data, collects words from the documents, calculates the norm of animal vectors, the norm of disease vectors, and the norm of collected word vectors based on the collected words and historical data, and calculates the similarity between words and animals, the similarity between words and diseases, and the word frequency; The criticality unit is used to receive the document and word frequency in the data similarity unit, and calculate the criticality value of the word in the document using the inverse word frequency. When the criticality value is greater than the set criticality value, it means that the word is critical in the document, and the word is named as a keyword; The important association unit is used to receive the keywords and word inverse document frequencies in the documents and criticality units in the data similarity unit, record the number of keywords, the number of characters, and the position of the sentences in the documents, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by analyzing the importance of the sentences and the average inverse document frequency value of the sentences, and analyze the relevance of adjacent sentences; When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation; When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not very similar in importance, and when the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation, and the corresponding sentence of the importance of the analyzed sentence is input into the Transformer model to output the key feature information of the sentence; The analysis and determination unit is used to receive key feature information of adjacent sentences or sentences in important associated units, integrate the key feature information of adjacent sentences or sentences into key documents, perform cluster analysis based on the key documents and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis.

[0005] As a further improvement of the technical solution, the data similarity unit includes a data calculation module and a similarity module; The data calculation module investigates data related to diseased animals through multiple channels and obtains historical data, then divides the data related to diseased animals into multiple document data, records the number of documents, and then collects words from the documents, inputs the animals, diseases and collected words into the Transformer model in big data technology, and the Transformer model outputs corresponding animal vectors, disease vectors and collected word vectors, and uses the animal vectors, disease vectors and collected word vectors to calculate the norm of the animal vector, the norm of the disease vector and the norm of the collected word vector.

[0006] As a further improvement of the technical solution, the similarity module is used to receive the animal vector, the disease vector, the collected word vector and the norm of the animal vector, the norm of the disease vector, the norm of the collected word vector, and the document in the data calculation module, and calculate the similarity between the word and the animal; When the similarity between words and animals is close , indicating that the two words "word" and "animal" are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. Use the number of times the word appears in the document and the total number of words in the document to calculate the word frequency. When the similarity between words and animals is close When , it indicates that there is no semantic correlation between the word and the word animal, and then the similarity between the word and the disease is calculated using the disease vector, the collected word vector, the norm of the disease vector, and the norm of the collected word vector; When the similarity between words and diseases is close , indicating that the two words "word" and "disease" are semantically related. Then the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word are recorded. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document.

[0007] As a further improvement of the technical solution, the criticality unit is used to receive the number of documents in the data calculation module, the number of documents containing words in the documents and similarity module, and the word frequency, and calculate the word inverse document frequency according to the number of documents and the number of documents containing words. , and then use the inverse document frequency and word frequency to calculate the criticality value of the word in the document When the criticality value is greater than the set criticality value, it means that the word is critical in the document and the word is named as a keyword.

[0008] As a further improvement of the technical solution, the important association unit includes an importance module and a relevance module; The importance module is used to receive keywords in the document and criticality unit in the data calculation module, record the number of keywords, the number of characters, and the position of the sentence in the document, input the number of keywords, the number of characters, and the position into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight, and uses the corresponding weight coefficient, position weight, number of keywords, and number of characters to analyze the importance of the sentence.

[0009] As a further improvement of the technical solution, the importance of analyzing the sentence in the importance module is implemented as follows: Collect the corresponding weight coefficients , , Position weight , Number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include , ,..., , , the specific algorithm formula is: .

[0010] As a further improvement of the technical solution, the relevance module is used to receive the importance of the analysis statement in the importance module, the number of keywords and the inverse document frequency of words in the key degree unit, and obtain the importance of the analysis statement from the importance of the analysis statement. The importance of analyzing statements , using the importance of analyzing sentences The importance of analyzing statements Calculate the change amount of adjacent sentences and the normalized variables of the high importance of adjacent sentences, analyze the relevance of adjacent sentences according to the change amount of adjacent sentences and the normalized variables of the high importance of adjacent sentences, and obtain the analyzed relevance of adjacent sentences; When the change of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation, and the importance of the analyzed sentences is The importance of analyzing statements The corresponding adjacent statement and statements The input is sent to the analysis and determination unit; When the change of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. When the correlation of the analyzed adjacent sentences exceeds the set stability range, it is considered that the adjacent sentences have weak correlation. The importance of analyzing statements After comparison, the sentences with lower importance are input into the Transformer model. The Transformer model outputs the key feature information of the sentences, and the sentences with higher importance and the key feature information of the sentences are input into the analysis determination unit.

[0011] As a further improvement of the technical solution, the analysis and determination unit is used to receive the corresponding adjacent statements in the relevance module. and statements Or the corresponding higher sentences and key feature information of the sentences, all the corresponding adjacent sentences and statements Or the corresponding relatively high sentences and the key feature information of the sentences are input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. Cluster analysis is performed using the extracted key data and key documents, and then animal diseases are identified based on the results of the cluster analysis.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. In the veterinary epidemiological survey and analysis system based on big data technology, the criticality unit calculates the inverse document frequency of the word according to the number of documents and the number of documents containing the word, and then uses the inverse document frequency and the word frequency to calculate the criticality value of the word in the document. When the criticality value is greater than the set criticality value, it means that the word is key in the document, and the word is named as a keyword. The criticality can filter out frequently appearing useless words, and avoid the frequent appearance of useless words being mistakenly regarded as key information, highlighting the truly important information, and improving the efficiency of extracting important information.

[0013] 2. In the veterinary epidemiological survey and analysis system based on big data technology, the importance module records the number of keywords and characters in each sentence in the document, as well as the position of the sentence in the document. The number of keywords, the number of characters and the position are input into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight. The importance of the sentence is analyzed using the corresponding weight coefficient, position weight, number of keywords and number of characters. Through sentence importance analysis, those insignificant or low-importance sentences can be effectively filtered out, the number of processed sentences can be reduced, the time for data processing and analysis can be significantly shortened, and processing efficiency can be improved.

[0014] 3. In the veterinary epidemiological survey and analysis system based on big data technology, the relevance module calculates the average inverse frequency value of the sentence according to the number of keywords and the inverse document frequency of the words, and then takes the sentence inverse frequency value from the average inverse frequency value of the sentence, calculates the change in the inverse frequency value of the adjacent sentence according to the sentence inverse frequency value, and normalizes the importance of the analyzed sentence with the high importance of the adjacent sentence. The relevance of the adjacent sentences is analyzed according to the change in the inverse frequency value of the adjacent sentence and the normalized variable of the high importance of the adjacent sentence, and the relevance of the analyzed adjacent sentences is obtained. The sentence inverse frequency can be used to identify those sentences that appear less frequently in the entire document collection but are important, so as to avoid mistakenly excluding important sentences that appear less frequently in the document, thereby improving the accuracy of the retrieval of important sentences. At the same time, the relevance of adjacent sentences is analyzed based on the sentence inverse frequency, and the key feature information is extracted for the sentences with low relevance, thereby reducing the number of sentences with low relevance that need to be processed, reducing the computational complexity, and improving the processing speed, especially in large-scale text processing and high-concurrency tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A unit block diagram of the present invention; Figure 2 It is a block diagram of a module unit of the present invention; Figure 3 A block diagram of a data similarity unit of the present invention; Figure 4 It is a block diagram of important associated units of the present invention.

[0016] The meaning of each number in the figure is: 1. Data similarity unit; 11. Data calculation module; 12. Similarity module; 2. Criticality unit; 3. Important related unit; 31. Importance module; 32. Relevance module; 4. Analysis and determination unit. DETAILED DESCRIPTION

[0017] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The present invention provides a veterinary epidemiology survey and analysis system based on big data technology, please refer to Figure 1 , including data similarity unit 1, criticality unit 2, important association unit 3, analysis and determination unit 4; The data similarity unit 1 investigates data related to diseased animals through multiple channels and obtains historical data, divides the investigated data related to diseased animals into multiple document data, collects words from the documents, calculates the norm of the animal vector, the norm of the disease vector, and the norm of the collected word vector based on the collected words and historical data, and calculates the similarity between the word and the animal, the similarity between the word and the disease, and the word frequency; The criticality unit 2 is used to receive the document and word frequency in the data similarity unit 1, and calculate the criticality value of the word in the document by using the inverse word frequency. When the criticality value is greater than the set criticality value, the word is named as a keyword; The important association unit 3 is used to receive the keywords and word inverse document frequencies in the document in the data similarity unit 1 and the criticality unit 2, record the number of keywords, the number of characters, and the position of the sentences in the document, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by analyzing the importance of the sentences and the average inverse document frequency value of the sentences, and analyze the correlation of adjacent sentences; When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation; When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not very similar in importance, and when the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation, and the corresponding sentence of the importance of the analyzed sentence is input into the Transformer model to output the key feature information of the sentence; The analysis and determination unit 4 is used to receive the key feature information of adjacent sentences or sentences in the important association unit 3, integrate the key feature information of adjacent sentences or sentences into key documents, and then perform cluster analysis based on the key documents and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis.

[0019] The following is a refinement of the above units, see Figure 2-Figure 4 ; The data similarity unit 1 includes a data calculation module 11 and a similarity module 12; Since animals may get sick during the process of raising them in farms, in order to prevent the sick animals from causing harm to other animals or humans, the data calculation module 11 is used to investigate the data related to the sick animals through multiple channels and obtain historical data, and then divide the investigated data related to the sick animals into multiple document data, and record the number of documents. , and then collect words from the document, input the animals, diseases and collected words into the Transformer model in big data technology, and the Transformer model outputs the corresponding animal vector , Disease Vector And the collected word vectors , using animal vectors , Disease Vector And the collected word vectors Calculate the norm of the animal vector , the norm of the disease vector and the norm of the collected word vectors ,in, Refers to the Animal vector , Refers to the Animal vector , Refers to the Disease vector , Refers to the Disease vector , Refers to the The collected word vectors , Refers to the The collected word vectors ,By calculating the norm of animal vectors, disease vectors, and related word vectors, the similarities between them can be quantified, which helps identify potential correlations; The data related to sick animals investigated include case records from veterinary clinics, environmental data from farms, and data on animal physiological indicators; Historical data includes key terms for animals and diseases; The similarity module 12 is used to receive the animal vector in the data calculation module 11 , Disease Vector , collected word vectors and the norm of the animal vector , the norm of the disease vector , the norm of the collected word vectors , documents, using animal vectors , collected word vectors and the norm of the animal vector , the norm of the collected word vectors Calculate the similarity between words and animals , where the value range is ,Through similarity analysis, it is possible to identify that the collected words are highly related to the specific physiological, pathological or disease characteristics of animals, and discover new research directions, which is convenient for improving the accuracy of determining the types of animal diseases; When words and animals are similar near , indicating that the word and the word animal are semantically related, and then record the number of times the word appears in the document. and the total number of words in the document , and the number of documents containing the term , using the number of times a word appears in a document and the total number of words in the document Calculate word frequency , which is convenient for the subsequent calculation of the inverse document frequency of words; When words and animals are similar near , indicating that there is no semantic correlation between the word and the word animal. Then, the disease vector , collected word vectors and the norm of the disease vector , the norm of the collected word vectors Calculate the similarity between words and diseases ,Through similarity analysis, it is possible to identify that the collected words are highly related to the specific physiological, pathological or disease characteristics of animals, and discover new research directions, which is convenient for improving the accuracy of determining the types of animal diseases; When the similarity between words and diseases near , indicating that the word and the disease are semantically related, and then record the number of times the word appears in the document. and the total number of words in the document , and the number of documents containing the term , using the number of times a word appears in a document and the total number of words in the document Calculate word frequency , which is convenient for the subsequent calculation of the inverse document frequency of words; The criticality unit 2 is used to receive the number of documents in the data calculation module 11 , documents and the number of documents containing the word in the similarity module 12 , word frequency or word frequency , according to the number of documents and the number of documents containing the term Calculate the inverse document frequency of a term , where if the number of documents containing the word When , it means that there is no word in all documents. When the inverse document frequency of a word is greater than the set value, it means that the word is not common in the document and is critical in the document. , which can identify those words that appear less frequently in the entire document collection but are key words, avoid mistakenly excluding key words that appear less frequently in the document, improve the accuracy of key word retrieval, and then use the word inverse document frequency and word frequency or word frequency Calculate the criticality of words in a document or , when the criticality value Greater than the set criticality value When the word is key in the document, it indicates that the word is key and the word is named as a keyword. The key degree can filter out useless frequently appearing words and avoid frequent words being mistakenly regarded as key information, highlighting the truly important information and improving the efficiency of extracting important information. Suppose we have 100 documents and consider the following two words: Term A appears in 10 documents, so : Term B appears in 2 documents, so : Word B The value is higher than that of word A, and word B is less common in the document, which means that word B has higher discrimination and information content and can more effectively indicate the main topic of the document containing it.

[0020] The important association unit 3 includes an importance module 31 and a relevance module 32; The importance module 31 is used to receive the document in the data calculation module 11 and the keywords in the criticality unit 2, and record the number of keywords in each sentence in the document. and the number of characters , and the position of the statement in the document , the number of keywords , Number of characters and location Input into the Transformer model, the Transformer model outputs the corresponding weight coefficient , and position weight , using the corresponding weight coefficient , , Position weight , Number of keywords and the number of characters Analyze the importance of the statement. When the importance of the analyzed statement is greater than the set importance value, it means that the importance value of the statement is high. Through statement importance analysis, those insignificant or low-importance statements can be effectively filtered out, reducing the number of processed statements, significantly shortening the time for data processing and analysis, and improving processing efficiency. Commands with high importance values ​​are passed to the relevance module 32; The principle of analyzing the importance of statements: Collect the corresponding weight coefficients , , Position weight , Number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include , ,..., , , the specific algorithm formula is: ; in, , There are two weight coefficients used to adjust the number of keywords and position weight Through sentence importance analysis, we can effectively filter out those insignificant or low-importance sentences, which helps reduce noise and make the truly important information more prominent; The relevance module 32 is used to receive the importance of the analysis statement in the importance module 31 , Number of keywords and the inverse document frequency of the words in criticality unit 2 , according to the number of keywords and term inverse document frequency Calculate the average reverse frequency value of the statement ,in, Refers to the The inverse document frequency of each word, and then the average inverse document frequency value of the sentence Take Statement and statements The frequency value of the statement reverse and , according to the statement reverse frequency value and Calculate the change in the reverse frequency value of adjacent sentences , and then analyze the importance of the sentence Get the importance of the analysis statement The importance of analyzing statements , will analyze the importance of the statement The importance of analyzing statements Normalize variables with high importance of adjacent sentences and , according to the change in the reverse frequency value of the adjacent sentence Normalized variables with high importance of adjacent sentences , Analyze the relevance of adjacent sentences and obtain the relevance of adjacent sentences analyzed , through the reverse frequency of sentences, we can identify those sentences that appear less frequently in the entire document collection but are important, avoid erroneous exclusion of important sentences that appear less frequently in the document, and improve the accuracy of important sentence retrieval. At the same time, based on the reverse frequency of sentences, we analyze the relevance of adjacent sentences, extract key feature information from sentences with low relevance, reduce the number of sentences with low relevance that need to be processed, and reduce the computational complexity. In particular, in large-scale text processing and high-concurrency tasks, the processing speed is improved; When the frequency value of the adjacent statement reverses Within the set difference range, it means that the adjacent sentences are very similar in importance. If the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation. The importance of analyzing statements The corresponding adjacent statement and statements The input is sent to the analysis and determination unit 4; When the frequency value of the adjacent statement reverses If the difference is beyond the set range, it means that the adjacent sentences are not similar in importance. If the correlation of the analyzed adjacent sentences is beyond the set stability range, the adjacent sentences are considered to have weak correlation. The importance of analyzing statements Compare and analyze the importance of the statement The sentences with relatively low correspondence are input into the Transformer model, and the Transformer model outputs the key feature information of the sentences, and the sentences with relatively high correspondence and the key feature information of the sentences are input into the analysis and determination unit 4; The analysis and determination unit 4 is used to receive the corresponding adjacent sentences in the relevance module 32 and statements Or the corresponding higher sentences and key feature information of the sentences, all the corresponding adjacent sentences and statements Or the corresponding relatively high sentences and the key feature information of the sentences are input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. By analyzing the data associated with the key documents, the key factors affecting animal health can be identified, providing accurate data for subsequent research, and cluster analysis is performed using the extracted key data and key documents. Then, based on the results of the cluster analysis, animal diseases are identified. Through cluster analysis of key data and key documents, the key variables related to specific diseases can be accurately identified, thereby improving the accuracy of disease identification.

[0021] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. Veterinary epidemiological survey and analysis system based on big data technology, characterized by: It includes data similarity unit (1), criticality unit (2), important association unit (3), and analysis and determination unit (4); The data similarity unit (1) investigates data related to diseased animals through multiple channels and obtains historical data, divides the investigated data related to diseased animals into multiple document data, collects words from the documents, calculates the norm of the animal vector, the norm of the disease vector, and the norm of the collected word vector based on the collected words and historical data, and calculates the similarity between the word and the animal, the similarity between the word and the disease, and the word frequency; The criticality unit (2) is used to receive the document and word frequency in the data similarity unit (1), and calculate the criticality value of the word in the document using the inverse word frequency. When the criticality value is greater than the set criticality value, it indicates that the word is critical in the document, and the word is named as a keyword; The important association unit (3) is used to receive the key words and word inverse document frequencies in the document in the data similarity unit (1) and the key degree unit (2), record the number of key words, the number of characters and the position of the sentences in the document, analyze the importance of the sentences and calculate the average inverse document frequency value of the sentences, calculate the change amount of adjacent sentences and the normalized variable of the high importance of adjacent sentences by analyzing the importance of the sentences and the average inverse document frequency value of the sentences, and analyze the relevance of adjacent sentences; When the change in the reverse frequency value of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation; When the change in the reverse frequency value of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not very similar in importance, and when the correlation of the analyzed adjacent sentences exceeds the set stability range, the adjacent sentences are considered to have weak correlation, and the corresponding sentence of the importance of the analyzed sentence is input into the Transformer model to output the key feature information of the sentence; The analysis and determination unit (4) is used to receive adjacent sentences or key feature information of sentences in the important association unit (3), integrate the adjacent sentences or key feature information of sentences into a key document, perform cluster analysis based on the key document and veterinary epidemiology, and identify animal diseases based on the results of the cluster analysis.

2. The veterinary epidemiology survey and analysis system based on big data technology according to claim 1, characterized in that: The data similarity unit (1) comprises a data calculation module (11) and a similarity module (12); The data calculation module (11) investigates data related to diseased animals through multiple channels and obtains historical data, then divides the investigated data related to diseased animals into multiple document data, records the number of documents, and then collects words from the documents, inputs the animals, diseases and collected words into a Transformer model in big data technology, and the Transformer model outputs corresponding animal vectors, disease vectors and collected word vectors, and uses the animal vectors, disease vectors and collected word vectors to calculate the norm of the animal vector, the norm of the disease vector and the norm of the collected word vector.

3. The veterinary epidemiology survey and analysis system based on big data technology according to claim 2 is characterized by: The similarity module (12) is used to receive the animal vector, the disease vector, the collected word vector and the norm of the animal vector, the norm of the disease vector, the norm of the collected word vector, and the document in the data calculation module (11), and calculate the similarity between the word and the animal; When the similarity between words and animals is close , indicating that the two words "word" and "animal" are semantically related. Then record the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word. Use the number of times the word appears in the document and the total number of words in the document to calculate the word frequency. When the similarity between words and animals is close When , it indicates that there is no semantic correlation between the word and the word animal, and then the similarity between the word and the disease is calculated using the disease vector, the collected word vector, the norm of the disease vector, and the norm of the collected word vector; When the similarity between words and diseases is close , indicating that the two words "word" and "disease" are semantically related. Then the number of times the word appears in the document, the total number of words in the document, and the number of documents containing the word are recorded. The word frequency is calculated using the number of times the word appears in the document and the total number of words in the document.

4. The veterinary epidemiology survey and analysis system based on big data technology according to claim 3 is characterized by: The criticality unit (2) is used to receive the number of documents in the data calculation module (11), the number of documents containing words in the documents and similarity module (12), and the word frequency, and calculate the word inverse document frequency according to the number of documents and the number of documents containing words. , and then use the inverse document frequency and word frequency to calculate the criticality value of the word in the document When the criticality value is greater than the set criticality value, it means that the word is critical in the document and the word is named as a keyword.

5. The veterinary epidemiology survey and analysis system based on big data technology according to claim 4 is characterized by: The important association unit (3) comprises an importance module (31) and a relevance module (32); The importance module (31) is used to receive keywords in the document and the criticality unit (2) in the data calculation module (11), record the number of keywords, the number of characters, and the position of the sentence in the document, input the number of keywords, the number of characters, and the position into the Transformer model, and the Transformer model outputs the corresponding weight coefficient and position weight, and uses the corresponding weight coefficient, position weight, number of keywords, and number of characters to analyze the importance of the sentence.

6. The veterinary epidemiology survey and analysis system based on big data technology according to claim 5 is characterized by: The importance module (31) analyzes the importance of a sentence in accordance with the following principles: Collect the corresponding weight coefficients , , Position weight , Number of keywords and the number of characters Calculate the importance of the statement. The importance of the statement is the importance of the analysis statement. , analyze the importance of the sentence include , ,..., , , the specific algorithm formula is: 。 7. The veterinary epidemiology survey and analysis system based on big data technology according to claim 5 is characterized by: The relevance module (32) is used to receive the importance of the sentence analyzed in the importance module (31), the number of keywords and the word inverse document frequency in the key degree unit (2), and calculate the average inverse document frequency value of the sentence according to the number of keywords and the word inverse document frequency. ,in, Refers to the The inverse document frequency of each word, and then take the sentence from the average inverse document frequency value and statements The frequency value of the statement reverse and , calculate the change in the reverse frequency value of adjacent sentences based on the reverse frequency value of the sentence , and then obtain the importance of the analysis statement from the importance of the analysis statement The importance of analyzing statements , will analyze the importance of the statement The importance of analyzing statements Normalize variables with high importance of adjacent sentences and , according to the change in the reverse frequency value of the adjacent sentence Normalized variables with high importance of adjacent sentences , Analyze the relevance of adjacent sentences and obtain the relevance of adjacent sentences analyzed ; When the change of adjacent sentences is within the set difference range, it means that the adjacent sentences are very similar in importance, and when the correlation of the analyzed adjacent sentences is within the set stability range, it is considered that the adjacent sentences have a strong correlation, and the importance of the analyzed sentences is The importance of analyzing statements The corresponding adjacent statement and statements The input is sent to the analysis and determination unit (4); When the change of adjacent sentences exceeds the set difference range, it means that the adjacent sentences are not similar in importance. When the correlation of the analyzed adjacent sentences exceeds the set stability range, it is considered that the adjacent sentences have weak correlation. The importance of analyzing statements After comparison, the sentences with lower importance are input into the Transformer model. The Transformer model outputs the key feature information of the sentences, and the sentences with higher importance and the key feature information of the sentences are input into the analysis and determination unit (4).

8. The veterinary epidemiology survey and analysis system based on big data technology according to claim 7 is characterized by: The analysis and determination unit (4) is used to receive the corresponding adjacent sentences in the relevance module (32) and statements Or the corresponding higher sentences and key feature information of the sentences, all the corresponding adjacent sentences and statements Or the corresponding relatively high sentences and the key feature information of the sentences are input into the Transformer model, the Transformer model is integrated into key documents, and then the association rule mining in big data technology is used to perform association analysis based on the key documents and veterinary epidemiology, and the key data associated with the key documents are extracted from veterinary epidemiology. Cluster analysis is performed using the extracted key data and key documents, and then animal diseases are identified based on the results of the cluster analysis.

Images