Food safety risk assessment system based on big data analysis

Abstract

The invention belongs to the technical field of food safety data analysis, and provides a food safety risk assessment system based on big data analysis, and the system comprises a data collection module which collects food safety related data from multiple channels; the data processing module performs preprocessing operation on the original data by using a big data platform; the data analysis module evaluates and predicts food safety risks by using a machine learning algorithm; the risk assessment module constructs a food safety risk assessment model in combination with historical data and real-time data; the early warning response module sets a threshold value, automatically triggers early warning and establishes a quick response mechanism; the interaction module is used for displaying the risk assessment result in a chart form; according to the method, food safety risks are assessed and predicted, text data in social media and news reports are analyzed, potential food safety problems are identified, comprehensive analysis is carried out in combination with multiple data types such as images, texts and sensor data, and the comprehensiveness of risk assessment is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of food safety data analysis, and specifically is a food safety risk assessment system based on big data analysis. Background Art

[0002] With the rapid development of globalization and the increasing complexity of the food supply chain, food safety issues have become increasingly prominent. In recent years, frequent food safety incidents have not only seriously affected the health of consumers, but also brought tremendous pressure to related companies and government agencies. Therefore, it is particularly important to establish an efficient and accurate food safety risk assessment system. Existing food safety risk assessments mainly rely on traditional laboratory testing and government supervision methods. This method has the disadvantages of long response time, high cost, and poor adaptability to emergencies.

[0003] At present, some data analysis-based technologies have begun to be applied in the field of food safety risk assessment. For example, data mining and pattern recognition using machine learning algorithms have been used to predict food safety risks. However, these systems usually rely on only a single type of data, such as historical inspection records or production logs, and lack the ability to conduct comprehensive analysis of multiple data sources. In addition, many existing technologies are inadequate in processing social media dynamic information and real-time news reports, resulting in the inability to timely identify the public's high concern about food safety and potential risks;

[0004] In addition, most existing risk assessment models are linear models that cannot effectively consider the impact of multi-dimensional factors such as different food categories, production processes, and seasonal changes, resulting in insufficient accuracy and reliability of assessment results. These defects make traditional risk assessment methods incapable of coping with complex and changing food safety environments.

[0005] To this end, technicians in this field have proposed a food safety risk assessment system based on big data analysis, which aims to integrate multiple data types, improve the comprehensiveness and timeliness of the assessment, provide more valuable information support for food safety management, and improve the accuracy and response speed of food safety risk assessment. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a food safety risk assessment system based on big data analysis to solve the problems raised in the background technology.

[0007] A food safety risk assessment system based on big data analysis, comprising:

[0008] The data collection module collects food safety-related data from multiple channels and uses IoT technology to monitor key indicators in the production, storage, transportation and sales of food in real time through sensors and RFID tags;

[0009] The data processing module uses the big data platform to pre-process the raw data, establish a data warehouse, and classify and index the data;

[0010] The data analysis module uses machine learning algorithms to assess and predict food safety risks, applies natural language processing technology to analyze text data in social media and news reports to identify potential food safety issues, tracks the trends and patterns of food safety incidents through time series analysis, and uses geographic information system (GIS) technology to analyze food safety incidents and geographic locations to identify high-risk areas;

[0011] The risk assessment module combines historical data and real-time data to build a food safety risk assessment model, taking into account factors such as different food categories, production processes and seasonal changes to establish a multi-dimensional risk assessment system;

[0012] The early warning response module sets a threshold. When the risk assessment result exceeds the preset safety threshold, an early warning is automatically triggered, and a rapid response mechanism is established. When food safety issues are found, relevant departments and enterprises are immediately notified to take measures.

[0013] The interactive module is used to display risk assessment results in the form of charts, generate customized reports, and provide the function of querying food safety risk information through an interactive interface.

[0014] Preferably, the data collection module collects food safety related data from multiple channels including government regulatory agencies, food production enterprises, supermarkets, and farmers' markets, and collects public concerns and complaints about food safety through social media, news reports, and consumer feedback.

[0015] Preferably, a stream processing framework is introduced into the data processing module to quickly process and respond to real-time data, and a data lake is established to store structured and unstructured data.

[0016] Preferably, the data analysis module uses a support vector machine to evaluate and predict food safety risks, and selects features related to food safety risks, including temperature, humidity, production date, and additive content; the data set is divided into a training set and a test set in a ratio of 7:3, and the training set is used to train the SVM model. For a linearly separable data set, the optimization problem of the SVM is expressed as:

[0017]

[0018] subject to you i (w·x i +b)≥1,i=1,2,...,N

[0019] Among them, w represents the weight vector, b represents the bias, and x i represents the feature vector of the i-th sample, y i Represents the label of the i-th sample; for nonlinearly separable data sets, a slack variable ξ is introduced i and penalty parameter C, the optimization problem is expressed as:

[0020]

[0021] subject to you i (w·x i +b)≥1-ξ,ξ i ≥0,i=1,2,...,N

[0022] To handle nonlinear problems, SVM uses the kernel function φ(x) to map the input space to a high-order feature space.

[0023] Preferably, the data analysis module also applies natural language processing (NIP) technology to analyze text data in social media and news reports. The original social media and news report texts are first cleaned and converted to remove HTML tags, special symbols, and stop words, and the text is converted to lowercase and word segmentation is performed. The text is represented as a vector using a bag-of-words model, and each word is regarded as an independent feature. The bag-of-words model is represented as follows: X = [x1, x2, ..., x n ], where x i Indicates the number of times the i-th word appears in the text; then use the TextBlob library to perform vocabulary-based sentiment analysis: Sentiment = TexBlob (text).sentiment, and then use latent Dirichlet allocation to find hidden topic distribution from a large amount of text, expressed as:

[0024]

[0025] Among them, w ij represents the jth word in the i-th document, z ij represents the jth topic of the ith document, θ represents the probability distribution of the topic, φ represents the probability distribution of the vocabulary, K represents the number of topics, V represents the number of vocabulary, and α represents the smoothing parameter of the new topic; then the BERT model of Hugging Face is used for public opinion monitoring, which is expressed as: Opinion = pipeline ("sentiment-analysis"), which monitors the changes in public opinion in social media and news reports in real time and identifies potential food safety issues.

[0026] Preferably, the data analysis module tracks the occurrence trend and pattern of food safety incidents through time series analysis, collects data on historical food safety incidents, including the time, location, and type of the incidents, and uses an autoregressive moving average time series model to describe the occurrence trend and pattern of food safety incidents, expressed as:

[0027]

[0028] Among them, X t represents the observed value at the tth time point, c represents the constant term, φ i represents the autoregressive coefficient, θ j represents the moving average coefficient, ε t Represents white noise.

[0029] Preferably, the data analysis module also uses geographic information system GIS technology to analyze food safety incidents and geographical locations, integrates food safety incident data with geographic information data, establishes a spatial database, and uses density-based clustering algorithm DBSCAN to describe the spatial distribution characteristics of food safety incidents, expressed as:

[0030] D(X,ε,MinPts)={C0,C1,...C k}

[0031] Among them, X represents the data set, ε represents the neighborhood radius, MinPts represents the minimum number of points, and C i Represents clusters, and based on the results of the risk assessment model, high-risk areas are identified.

[0032] Preferably, the risk assessment module combines historical data and real-time data by weighted average, expressed as: fused =αX historical +(1-α)X real-time , where α represents the weight parameter, X historical , X real-time Represent the feature matrices of historical data and real-time data respectively; and then build multiple decision trees to improve the accuracy and stability of the model, expressed as: Where N represents the number of decision trees, h i (x) represents the prediction result of the i-th decision tree.

[0033] Preferably, the risk assessment module further collects data from different food categories, production processes and seasonal changes, extracts key features, fuses data from different dimensions to form a comprehensive data set, standardizes the data to make the dimensions of different features consistent, and combines data from different dimensions through weighted averaging, expressed as:

[0034] X fused =α1Xcategory +α2X process +α3X season

[0035] Among them, α1, α2, α3 represent weight parameters, X category , X process , X season Represent the food category, production process and seasonal change feature matrices respectively.

[0036] Preferably, the early warning response module determines the key indicators of food safety based on historical data, and sets the safety threshold of each key indicator based on regulatory standards and risk assessment results. The collected data is compared with the preset safety threshold through real-time monitoring. When an indicator exceeds the preset safety threshold, the system automatically triggers an early warning and generates early warning information.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The present invention uses support vector machines to evaluate and predict food safety risks, improve the accuracy of predictions, and analyze text data in social media and news reports to identify potential food safety issues and promptly discover hot issues of public concern. At the same time, it combines multiple data types such as images, text, and sensor data to conduct comprehensive analysis and improve the comprehensiveness of risk assessment.

[0039] 2. The present invention establishes a multi-dimensional risk assessment system by taking into account factors such as different food categories, production processes and seasonal changes, improves the accuracy and reliability of the assessment, and sets thresholds. When the risk assessment results exceed the preset safety threshold, an early warning is automatically triggered to promptly notify relevant departments and enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the structure of the food safety risk assessment system of the present invention. DETAILED DESCRIPTION

[0041] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0042] As attached Figure 1 As shown:

[0043] Embodiment 1: The present invention provides a food safety risk assessment system based on big data analysis, including: a data acquisition module, which collects food safety related data from multiple channels, and uses the Internet of Things technology to monitor key indicators in the production, storage, transportation and sales of food in real time through sensors and RFID tags; a data processing module, which uses a big data platform to pre-process the original data, establishes a data warehouse, and classifies and indexes the data; a data analysis module, which uses a machine learning algorithm to assess and predict food safety risks, applies natural language processing technology to analyze text data in social media and news reports, identifies potential food safety issues, and tracks the occurrence of food safety incidents through time series analysis. The system can also be used to identify food safety risks, trends and patterns, and use geographic information system (GIS) technology to analyze food safety incidents and geographical locations to identify high-risk areas; the risk assessment module combines historical data and real-time data to build a food safety risk assessment model, and takes into account factors such as different food categories, production processes and seasonal changes to establish a multi-dimensional risk assessment system; the early warning response module sets thresholds. When the risk assessment results exceed the preset safety threshold, an early warning is automatically triggered, and a rapid response mechanism is established. When food safety problems are found, relevant departments and companies are immediately notified to take measures; the interactive module is used to display the risk assessment results in the form of charts, generate customized reports, and provide the function of querying food safety risk information through an interactive interface.

[0044] The data collection module collects food safety related data from multiple channels including government regulatory agencies, food production enterprises, supermarkets, and farmers' markets, and collects public concerns and complaints about food safety through social media, news reports, and consumer feedback. The data collection module is responsible for collecting data from multiple sources, including government regulatory agencies, enterprises, supermarkets, farmers' markets, social media, and news reports, and then uses IoT technologies such as sensors and RFID tags to monitor key indicators in the production, storage, transportation, and sales of food in real time, and collects public concerns and complaints about food safety through social media, news reports, and consumer feedback.

[0045] A stream processing framework is introduced into the data processing module to quickly process and respond to real-time data, and a data lake is established to store structured and unstructured data, providing more flexible data management and analysis capabilities.

[0046] The data analysis module uses support vector machines to evaluate and predict food safety risks. It selects features related to food safety risks, including temperature, humidity, production date, and additive content. The data set is divided into a training set and a test set in a ratio of 7:3. The training set is used to train the SVM model, and the test set is used to evaluate the performance of the model. Based on the evaluation results, the model parameters are adjusted, cross-validation is performed, and the model performance is optimized. For linearly separable data sets, the optimization problem of SVM is expressed as:

[0047]

[0048] subject to you i (w·x i +b)≥1,i=1,2,...,N

[0049] Among them, w represents the weight vector, b represents the bias, and x i represents the feature vector of the i-th sample, y i Represents the label of the i-th sample; for nonlinearly separable data sets, a slack variable ξ is introduced i and penalty parameter C, the optimization problem is expressed as:

[0050]

[0051] subject to you i (w·x i +b)≥1-ξ,ξ i ≥0,i=1,2,...,N

[0052] To handle nonlinear problems, SVM uses the kernel function φ(x) to map the input space to a high-order feature space.

[0053] The data analysis module tracks the occurrence trends and patterns of food safety incidents through time series analysis. It collects data on historical food safety incidents, including the time, location, and type of the incidents, and uses an autoregressive moving average time series model to describe the occurrence trends and patterns of food safety incidents, expressed as:

[0054]

[0055] Among them, X t represents the observed value at the tth time point, c represents the constant term, φ i represents the autoregressive coefficient, θ j represents the moving average coefficient, ε t Represents white noise. The data analysis module combines multiple data types such as images, text, and sensor data for comprehensive analysis to improve the comprehensiveness of risk assessment.

[0056] The risk assessment module combines historical data and real-time data through weighted average, expressed as: fused =αX historical +(1-α)X real-time , where α represents the weight parameter, X historical , X real-time Represent the feature matrices of historical data and real-time data respectively; and then build multiple decision trees to improve the accuracy and stability of the model, expressed as: Where N represents the number of decision trees, h i (x) represents the prediction result of the i-th decision tree. The model is trained using historical data and real-time data, and the performance of the model is evaluated through cross-validation. Then, the trained model is used to predict future food safety incidents, and the prediction results are intuitively displayed in the form of charts, maps, etc. for easy understanding and reference by users.

[0057] The risk assessment module also collects data from different food categories, production processes, and seasonal changes, extracts key features, fuses data from different dimensions to form a comprehensive data set, standardizes the data to make the dimensions of different features consistent, and combines data from different dimensions through weighted averaging, expressed as:

[0058] X fused =α1X category +α2X process +α3X season

[0059] Among them, α1, α2, α3 represent weight parameters, X category , X process , X season They represent food categories, production processes, and seasonal change feature matrices respectively. Food categories include meat, vegetables, fruits, dairy products, etc.; production processes include data on food production, processing, storage, transportation, and sales; seasonal changes include environmental data in different seasons, such as temperature and humidity.

[0060] The early warning response module determines the key indicators of food safety based on historical data, and sets the safety threshold of each key indicator based on regulatory standards and risk assessment results. The collected data is compared with the preset safety threshold through real-time monitoring. When an indicator exceeds the preset safety threshold, the system automatically triggers an early warning and generates early warning information. Early warning information is immediately notified to relevant departments and enterprises through multiple channels such as SMS, email, and APP push. The early warning information includes detailed information such as the exceeded indicator, the degree of exceeding the standard, the time, and the location; in the subsequent processing, the excessive food is traced to find the root cause of the problem, and the corresponding control measures are taken. According to the cause of the problem, corrective measures are formulated and implemented to prevent similar problems from happening again.

[0061] Embodiment 2: This embodiment is basically the same as the previous embodiment, except that the data analysis module also applies natural language processing (NIP) technology to analyze text data in social media and news reports. The relevant text data is collected from social media platforms and news websites, and the data is obtained using API or web crawler tools and stored in a database. The original social media and news report texts are first cleaned and converted, HTML tags, special symbols, and stop words are removed, the text is converted to lowercase, and lexical segmentation is performed. The bag-of-words model is used to represent the text in vector form, and each word is regarded as an independent feature. The bag-of-words model is represented as: X = [x1, x2, ..., x n ], where x i Indicates the number of times the i-th word appears in the text; then use the TextBlob library to perform vocabulary-based sentiment analysis: Sentiment = TexBlob (text).sentiment, and then use latent Dirichlet allocation to find hidden topic distribution from a large amount of text, expressed as:

[0062]

[0063] Among them, w ij represents the jth word in the i-th document, z ij represents the jth topic of the ith document, θ represents the probability distribution of the topic, φ represents the probability distribution of the vocabulary, K represents the number of topics, V represents the number of vocabulary, and α represents the smoothing parameter of the new topic; then the BERT model of Hugging Face is used for public opinion monitoring, which is expressed as: Opinion = pipeline ("sentiment-analysis"), which monitors the changes in public opinion in social media and news reports in real time and identifies potential food safety issues.

[0064] From the above, we can see that by effectively using natural language processing technology to analyze text data in social media and news reports, we can identify potential food safety issues and provide scientific basis and technical support for food safety management.

[0065] Embodiment 3: This embodiment is basically the same as the previous embodiment, except that the data analysis module also uses geographic information system GIS technology to analyze food safety incidents and geographic locations, integrates food safety incident data with geographic information data, establishes a spatial database, and uses density-based clustering algorithm DBSCAN to describe the spatial distribution characteristics of food safety incidents, which is expressed as:

[0066] D(X,ε,MinPts)={C0,C1,...C k}

[0067] Among them, X represents the data set, ε represents the neighborhood radius, MinPts represents the minimum number of points, and C i Represents clusters, and based on the results of the risk assessment model, high-risk areas are identified.

[0068] From the above, we can see that by using GIS technology to analyze food safety incidents and geographic locations, high-risk areas can be identified and focused on for supervision.

[0069] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.

[0070] It will be appreciated that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will be a routine task of design, fabrication, and production for those of ordinary skill having the benefit of this disclosure without undue experimentation.

[0071] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A food safety risk assessment system based on big data analysis, characterized in that: include: The data collection module collects food safety-related data from multiple channels and uses IoT technology to monitor key indicators in the production, storage, transportation and sales of food in real time through sensors and RFID tags; The data processing module uses the big data platform to pre-process the raw data, establish a data warehouse, and classify and index the data; The data analysis module uses machine learning algorithms to assess and predict food safety risks, applies natural language processing technology to analyze text data in social media and news reports to identify potential food safety issues, tracks the trends and patterns of food safety incidents through time series analysis, and uses geographic information system (GIS) technology to analyze food safety incidents and geographic locations to identify high-risk areas; The risk assessment module combines historical data and real-time data to build a food safety risk assessment model, taking into account factors such as different food categories, production processes and seasonal changes to establish a multi-dimensional risk assessment system; The early warning response module sets a threshold. When the risk assessment result exceeds the preset safety threshold, an early warning is automatically triggered, and a rapid response mechanism is established. When food safety issues are found, relevant departments and enterprises are immediately notified to take measures. The interactive module is used to display risk assessment results in the form of charts, generate customized reports, and provide the function of querying food safety risk information through an interactive interface.

2. A food safety risk assessment system based on big data analysis as claimed in claim 1, characterized in that: The data collection module collects food safety related data from multiple channels including government regulatory agencies, food production enterprises, supermarkets, and farmers' markets, and collects public concerns and complaints about food safety through social media, news reports, and consumer feedback.

3. A food safety risk assessment system based on big data analysis as claimed in claim 1, characterized in that: A stream processing framework is introduced into the data processing module to quickly process and respond to real-time data, and a data lake is established to store structured and unstructured data.

4. A food safety risk assessment system based on big data analysis as claimed in claim 1, characterized in that: The data analysis module uses support vector machines to evaluate and predict food safety risks, and selects features related to food safety risks, including temperature, humidity, production date, and additive content; the data set is divided into a training set and a test set in a ratio of 7:3, and the training set is used to train the SVM model. For a linearly separable data set, the optimization problem of the SVM is expressed as: subject to y i (w·x i +b)≥1,i=1,2,...,N Among them, w represents the weight vector, b represents the bias, and x i represents the feature vector of the i-th sample, y i Represents the label of the i-th sample; for nonlinearly separable data sets, a slack variable ξ is introduced i and penalty parameter C, the optimization problem is expressed as: subject to y i (w·x i +b)≥1-ξ,ξ i ≥0,i=1,2,...,N To handle nonlinear problems, SVM uses the kernel function φ(x) to map the input space to a high-order feature space.

5. A food safety risk assessment system based on big data analysis as claimed in claim 4, characterized in that: The data analysis module also applies natural language processing (NIP) technology to analyze text data in social media and news reports. The original social media and news report texts are first cleaned and converted to remove HTML tags, special symbols, and stop words, and the text is converted to lowercase and lexical segmentation is performed. The bag-of-words model is used to represent the text in vector form, and each word is regarded as an independent feature. The bag-of-words model is represented as: X = [x1, x2, ..., x n ], where x i Indicates the number of times the i-th word appears in the text; Then use the TextBlob library to perform vocabulary-based sentiment analysis: Sentiment = TexBlob (text). Sentiment, and then use latent Dirichlet allocation to find hidden topic distribution from a large amount of text, expressed as: Among them, w ij represents the jth word in the i-th document, z ij represents the jth topic of the ith document, θ represents the probability distribution of the topic, φ represents the probability distribution of the vocabulary, K represents the number of topics, V represents the number of vocabulary, and α represents the smoothing parameter of the new topic; then the BERT model of Hugging Face is used for public opinion monitoring, which is expressed as: Opinion = pipeline ("sentiment-analysis"), which monitors the changes in public opinion in social media and news reports in real time and identifies potential food safety issues.

6. A food safety risk assessment system based on big data analysis as claimed in claim 5, characterized in that: The data analysis module tracks the occurrence trend and pattern of food safety incidents through time series analysis, collects data on historical food safety incidents, including the time, location, and type of the incident, and uses an autoregressive moving average time series model to describe the occurrence trend and pattern of food safety incidents, expressed as: Among them, X t represents the observed value at the tth time point, c represents the constant term, φ i represents the autoregressive coefficient, θ j represents the moving average coefficient, ε t Represents white noise.

7. A food safety risk assessment system based on big data analysis as claimed in claim 6, characterized in that: The data analysis module also uses geographic information system (GIS) technology to analyze food safety incidents and geographic locations, integrates food safety incident data with geographic information data, establishes a spatial database, and uses density-based clustering algorithm DBSCAN to describe the spatial distribution characteristics of food safety incidents, expressed as: D(X,ε,MinPts)={C0,C1,...C k } Among them, X represents the data set, ε represents the neighborhood radius, MinPts represents the minimum number of points, and C i Represents clusters, and based on the results of the risk assessment model, high-risk areas are identified.

8. A food safety risk assessment system based on big data analysis as claimed in claim 1, characterized in that: The risk assessment module combines historical data and real-time data through weighted average, expressed as: fused =αX historical +(1-α)X real-time , where α represents the weight parameter, X historical , X real-time Represent the feature matrices of historical data and real-time data respectively; and then build multiple decision trees to improve the accuracy and stability of the model, expressed as: Where N represents the number of decision trees, h i (x) represents the prediction result of the i-th decision tree.

9. A food safety risk assessment system based on big data analysis as claimed in claim 8, characterized in that: The risk assessment module also collects data from different food categories, production processes and seasonal changes, extracts key features, fuses data from different dimensions to form a comprehensive data set, standardizes the data to make the dimensions of different features consistent, and combines data from different dimensions through weighted averaging, expressed as: X fused =α1X category +α2X process +α3X season Among them, α1, α2, α3 represent weight parameters, X category , X process , X season Represent the food category, production process and seasonal change feature matrices respectively.

10. A food safety risk assessment system based on big data analysis as claimed in claim 1, characterized in that: The early warning response module determines the key indicators of food safety based on historical data, and sets the safety threshold of each key indicator based on regulatory standards and risk assessment results. The collected data is compared with the preset safety threshold through real-time monitoring. When an indicator exceeds the preset safety threshold, the system automatically triggers an early warning and generates early warning information.

Images