Food supervision and law enforcement system

Abstract

The invention relates to a food supervision and law enforcement system. The system comprises a data acquisition unit and a risk assessment unit. The data acquisition unit is used for acquiring first multi-modal data for a supervised merchant, and the first multi-modal data comprises a food image, an operation room video and public opinion data; the risk assessment unit is used for executing a first analysis step, a fast detection image acquisition step and a second analysis step; wherein the first analysis step comprises the steps of determining a first risk score time sequence of a supervised merchant in a future time period based on first multi-modal data respectively acquired in a plurality of periods; determining whether alarm information is sent based on the first risk score time sequence; a fast detection image acquisition step, wherein a fast detection image of a detection appliance in the to-be-on-site law enforcement merchant indicated in the alarm information is acquired; the second analysis step comprises the steps that a risk score is determined based on the first multi-modal data and the fast detection image, and the risk score represents the food risk degree. And the supervision accuracy and perspectiveness are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of food supervision, and particularly relates to a food supervision and law enforcement system. BACKGROUND

[0002] Food safety is a heavy weight of "people's livelihood bottom line" - it is both the basis for maintaining life and health, and is a measure of social civilization. The supervision of food safety of merchants is an important work to maintain the people's livelihood bottom line.

[0003] In the related art, data for evaluating food safety risks of merchants is single, and only food is detected, ignoring non-standard operation of food processing links. Moreover, external dynamic factors outside the merchants are not considered. This makes the evaluation result low in accuracy. In addition, it lacks foresight, limiting the defensive ability of food supervision. SUMMARY

[0004] Therefore, the present disclosure provides a food supervision and law enforcement scheme.

[0005] According to an aspect of the present disclosure, a food supervision and law enforcement scheme system is provided, which comprises:

[0006] The system comprises a data acquisition unit and a risk assessment unit.

[0007] The data acquisition unit is configured to acquire first multi-modal data for a supervised merchant, the first multi-modal data comprising at least food images showing the appearance of food, operation room videos showing the scene in the food processing place, and public opinion data showing consumer feedback information.

[0008] The risk assessment unit is configured to perform a first analysis step, a fast inspection image acquisition step, and a second analysis step.

[0009] The first analysis step comprises:

[0010] Based on the first multi-modal data acquired in each period, a first risk score time series of the supervised merchant in a future time period is determined.

[0011] Based on the first risk score time series, it is determined whether to issue an alarm information.

[0012] The fast inspection image acquisition step comprises:

[0013] Acquiring a fast inspection image of a detection tool in a to-be-on-site law enforcement merchant indicated in the alarm information.

[0014] The second analysis step comprises:

[0015] Determine a risk score based on the first multi-modal data and the fast inspection image, the risk score representing a degree of food risk.

[0016] In a possible implementation, the risk assessment unit comprises a portable detection terminal.

[0017] The portable detection terminal is configured to perform the first analysis step, the fast inspection image acquisition step, and the second analysis step.

[0018] The risk assessment unit comprises a portable detection terminal and an intelligent analysis platform.

[0019] The intelligent analysis platform is configured to perform the first analysis step and the second analysis step.

[0020] The portable detection terminal is configured to perform the fast inspection image acquisition step.

[0021] In a possible implementation, the risk assessment unit is further configured to:

[0022] Preprocess the first multi-modal data to obtain preprocessed data, the preprocessed data comprising text descriptions corresponding to the first multi-modal data and image labels.

[0023] Obtain weights corresponding to various types of sub-data in the preprocessed data.

[0024] Determine a first risk score based on the preprocessed data and the weights, and obtain a first risk score time sequence based on the first risk scores corresponding to the periods.

[0025] In a possible implementation, the risk assessment unit is further configured to:

[0026] Dynamically determine an alarm threshold based on food safety influencing factors, the food safety influencing factors at least comprising seasonal factors.

[0027] Compare the first risk score time sequence with the alarm threshold to obtain a comparison result, and issue an alarm information in a case where the comparison result meets an alarm condition.

[0028] In a possible implementation, the intelligent analysis platform is further configured to:

[0029] Generate a geographic heat map and an optimized route with a departure location as a starting point according to the geographic locations of the to-be-on-site-enforced vendors and the first risk score time sequences corresponding to the to-be-on-site-enforced vendors.

[0030] Send the geographic heat map and the optimized route.

[0031] In a possible implementation, the first multi-modal data further includes: an environmental parameter of the supervised merchant, and on-site feedback information.

[0032] In a possible implementation, the risk assessment unit is further configured to:

[0033] extracting keywords reflecting food safety issues for text data;

[0034] identifying food in image data to obtain food position labels, and identifying appearance features of the food in the image data to obtain food appearance description text for the image data;

[0035] identifying characters in the video to obtain character position labels, and identifying irregular actions of the characters to obtain irregular action description text containing time points.

[0036] In a possible implementation, the risk assessment unit is further configured to:

[0037] establishing a cross-modal correlation graph, the cross-modal correlation graph including a center node and an edge node connected to the center node, the center node representing a geographic location of the supervised merchant, the edge node representing sub-data in the preprocessed data, and an edge between the edge node and the center node carrying a weight corresponding to the sub-data;

[0038] inputting the cross-modal correlation graph into a large language model to obtain the first risk score.

[0039] In a possible implementation, the portable detection terminal is configured to add noise to the first multi-modal data and the quick detection image to obtain encrypted data, and send the encrypted data.

[0040] The intelligent analysis platform is configured to record the encrypted data in the form of a blockchain.

[0041] The intelligent analysis platform is configured to obtain an initial quick detection result based on the quick detection image.

[0042] The intelligent analysis platform is configured to process the initial quick detection result based on zero-knowledge proof technology to obtain a quick detection result, send the quick detection result, and use the risk score as a law enforcement reference.

[0043] The portable detection terminal is configured to receive and display the geographic heat map, the optimized route, and the warning information.

[0044] In a possible implementation, the portable detection terminal performs the second analysis step based on MobileNetV2 and performs the first analysis step based on a gated recurrent unit.

[0045] According to another aspect of the present disclosure, there is provided an electronic device comprising a memory, a processor, and a computer program stored on the memory, the processor executing the computer program to implement the functions of the above system.

[0046] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the functions of the above system.

[0047] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, or a non-transitory computer-readable storage medium carrying a computer program, the computer program being executed by a processor to implement the functions of the above system.

[0048] In the embodiments of the present disclosure, the first multi-modal data used not only contains food images, but also contains operation room videos and public opinion information. Moreover, based on the first multi-modal data obtained in multiple periods, the first risk score time series of the supervised business in the future time period is evaluated. Thus, the supervision accuracy and foresight are improved, and the supervision can be implemented in time when the risk has not occurred or has just occurred, and the accurate preventive supervision is truly achieved.

[0049] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0051] Figure 1 A structural schematic diagram of a food supervision and law enforcement system provided for an embodiment of the present disclosure.

[0052] Figure 2 A structural schematic diagram of a risk assessment unit provided for an embodiment of the present disclosure.

[0053] Figure 3 A structural schematic diagram of another risk assessment unit provided for an embodiment of the present disclosure.

[0054] Figure 4 A structural schematic diagram of an electronic device for controlling a food supervision and law enforcement system provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] Various exemplary embodiments, features, and aspects of the present disclosure are explained more fully with reference to the drawings. The same reference numbers in different drawings denote the same or similar functions in which they are found. Although the various aspects of the embodiments are illustrated and described in detail, they are not limited to the precise configuration shown and can be used with or in various other embodiments. Embodiments can take on various other forms without departing from the spirit of the inventive concept.

[0056] As used herein, the terms "comprise", "comprising", "have", "having", "include", "including", "contain", "containing", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but do not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof.

[0057] When an element is referred to as being "connected", "coupled", "responsive", or "related" to another element, it can be directly connected, coupled, responsive, or related to the other element, or intervening elements can be present.

[0058] Although the terms first, second, third, etc. can be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Thus, a first element / operation in some embodiments can be termed a second element / operation in other embodiments without departing from the teachings of the inventive concept.

[0059] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0060] In addition, for the purpose of convenience and brevity, detailed descriptions of well-known devices, methods, procedures, components and circuits have been omitted so as not to obscure the underlying principles of the present disclosure. Embodiments of the present disclosure are described herein with reference to specific language that will give practical effect to the teachings of the application. The language chosen is for the purposes of clarity and the present application is only meant to be limited as described in the claims and equivalents thereof.

[0061] Figure 1 A structural schematic diagram of a food supervision and law enforcement system is provided for the embodiments of the present disclosure. As shown in FIG. 1, the food supervision and law enforcement system comprises a food supervision and law enforcement platform 100, a food supervision and law enforcement database 200, and a food supervision and law enforcement terminal 300. Figure 1As shown, the system 20 comprises: a data acquisition unit 21 and a risk assessment unit 22; the data acquisition unit 21 is configured to acquire first multi-modal data for a supervised merchant, the first multi-modal data at least comprising: food images reflecting the appearance of food, operation room videos reflecting the scene in the food processing place, public opinion data reflecting consumer feedback information; the risk assessment unit 22 is configured to perform a first analysis step, a fast inspection image acquisition step, and a second analysis step; wherein the first analysis step comprises: determining a first risk score time series of the supervised merchant in a time period based on the first multi-modal data acquired in each period; determining whether to issue an alarm information based on the first risk score time series; the fast inspection image acquisition step comprises: acquiring a fast inspection image of a detection device in the supervised merchant indicated in the alarm information; the second analysis step comprises: determining a risk score based on the first multi-modal data and the fast inspection image, the risk score serving as a law enforcement reference.

[0062] The supervised merchant can be a merchant engaged in food processing, production, and sales. For example, the supervised merchant can be a restaurant, a food stall, a food processing place, etc. The first multi-modal data can include text data, image data, and video data. The food image can be an image reflecting the color and freshness of the food. The food image can be collected according to a preset first frequency. The operation room video can record the operation behavior of the food processing personnel and the environment in the operation room. The public opinion data can be the evaluation, feedback, and news media reports of the supervised merchant or food safety incidents on social media by consumers. The public opinion data can be text data. In the embodiments of the present disclosure, the food image and the public opinion data can be collected according to the first frequency, for example, once every 3 hours. The operation room video can be collected in real time.

[0063] In a possible implementation, the first multi-modal data can further comprise: environmental parameters of the supervised merchant, and on-site feedback information.

[0064] The environmental parameters can reflect the environmental conditions in the operation room. The environmental parameters can include the temperature, humidity, etc. in the operation room. The on-site feedback information can reflect the feedback of the law enforcement personnel and / or consumers on food safety issues of the supervised merchant on site. The on-site feedback information can be text data and image data. The on-site feedback information can include historical data. In the embodiments of the present disclosure, the environmental parameters can be collected according to the first frequency.

[0065] In this example, the first multimodal data further includes environmental parameters and on-site feedback information. Improving the comprehensiveness of the first multimodal data from the perspective of the operating room environment helps enhance the accuracy of food risk assessment for regulated merchants. The more objective and truthful on-site feedback information can effectively correct public opinion data, further improving the accuracy of food risk assessment for regulated merchants.

[0066] The period here can be a preset time period. Alternatively, the time interval between two or more collections of the first multimodal data can be considered as a period. For example, 24 hours can be used as a period. The future time period can be the time period after the process of determining the first risk score time series. For example, the future time period can be 48 hours or 72 hours after the process of determining the first risk score time series. The first risk score time series can represent the first risk score corresponding to future moments at equal intervals. The interval can be set as needed. For example, the interval can be 12 hours or 24 hours. The first risk score time series can reflect the food risk trend of the regulated merchants in the future time period.

[0067] In this embodiment of the disclosure, it can be determined whether to issue a warning message based on a first risk score time series.

[0068] For example, a risk trend can be determined within a future time period based on a first risk score time series, and based on this risk trend, it can be determined whether to issue an alarm. For instance, an alarm is issued if the risk trend is greater than a risk trend threshold, and no alarm is issued if the risk trend is not greater than the risk trend threshold.

[0069] For example, it can be determined whether to issue an alarm based on a first risk score contained in a first risk score time series. For instance, the first risk score can be compared to a risk score threshold; if the first risk score is greater than the risk score threshold, an alarm is issued. If the first risk score is not greater than the risk score threshold, no alarm is issued.

[0070] Alarm messages can indicate whether on-site enforcement should be carried out on regulated businesses. For ease of description, regulated businesses indicated in the alarm message as requiring on-site enforcement are named "Businesses Pending On-Site Enforcement." Rapid test images of testing equipment within the businesses awaiting on-site enforcement can be obtained during the enforcement process. Testing equipment can be test strips used to test food or the operating environment. Testing equipment can also be testing devices used to test food or the operating environment and displaying test results. Rapid test images are images taken of test strips or testing devices displaying test results.

[0071] Risk assessment unit 22 can determine the risk score of the merchant to be subject to on-site enforcement based on the rapid detection image and the first multimodal data. Here, all or part of the first multimodal data can be used to determine the risk score. For example, the risk score of the merchant to be subject to on-site enforcement can be determined using the first multimodal data acquired in the last cycle combined with the rapid detection image.

[0072] Risk scores can characterize the food safety risks of businesses identified during on-site enforcement. Risk scores can serve as a reference for determining enforcement measures.

[0073] In this embodiment, the first multimodal data used includes not only food images but also operational video and public opinion information. Furthermore, based on the first multimodal data acquired over multiple periods, a time series of the first risk score for regulated merchants is assessed for future timeframes. This improves the accuracy and foresight of regulation, enabling timely implementation of regulations before or immediately after a risk occurs, truly achieving accurate and defensive regulation.

[0074] In one possible implementation, the risk assessment unit 22 includes a portable detection terminal 221 and an intelligent analysis platform 222; the intelligent analysis platform 222 is used to execute the first analysis step and the second analysis step; the portable detection terminal 221 is used to execute the rapid image acquisition step; or, the risk assessment unit 22 includes a portable detection terminal 221; the portable detection terminal 221 is used to execute the first analysis step, the rapid image acquisition step, and the second analysis step.

[0075] Figure 2 This is a schematic diagram of the structure of a risk assessment unit provided in an embodiment of this disclosure. Figure 2 As shown, the data acquisition unit 21 can transmit the acquired first multimodal data. The intelligent analysis platform 222 receives and stores the first multimodal data. Furthermore, the intelligent analysis platform 222 can execute a first analysis step, namely, determining a first risk score time series. Based on the first risk score time series, it determines whether to issue an alarm message. If an alarm message is issued, the portable detection terminal 221 receives the alarm message. Law enforcement personnel, carrying the portable detection terminal 221, arrive at the merchant to be inspected and use the portable detection terminal 221 to acquire a rapid detection image. The portable detection terminal 221 transmits the rapid detection image. The intelligent analysis platform 222 receives the rapid detection image and, based on the first multimodal data and the rapid detection image, determines the detection result and risk score. The intelligent analysis platform 222 stores the rapid detection image, detection result, and risk score. The intelligent analysis platform 222 transmits the detection result and risk score. The portable detection terminal 221 receives the detection result and risk score for law enforcement personnel's reference.

[0076] In this embodiment, computationally intensive steps involving calculation and analysis are performed on the intelligent analysis platform, while lightweight computational steps such as rapid image acquisition and risk score display are performed on the portable detection terminal. The intelligent analysis platform offers greater physical space and performance support than the portable detection terminal; moreover, the intelligent analysis platform and the portable detection terminal can work together and divide tasks during the risk score determination process, improving the overall efficiency and accuracy of risk score determination.

[0077] Figure 3 This is a schematic diagram of the structure of another risk assessment unit provided in an embodiment of this disclosure. (See diagram below.) Figure 3 As shown, the portable detection terminal 221 can complete the first analysis step, the rapid image acquisition step, and the second analysis step. The intelligent analysis platform 222 is used to store the data generated and collected in these steps, as well as the first multimodal data collected by the data acquisition unit 21.

[0078] In this way, the portable testing terminal can determine the risk score even offline. Since there is no need to interact with the intelligent analysis platform during the risk score determination process, the risk of data tampering is reduced, thus improving the accuracy of the risk score.

[0079] In one possible implementation, the risk assessment unit is further configured to: preprocess the first multimodal data to obtain preprocessed data, the preprocessed data including text descriptions and image annotations corresponding to the first multimodal data; obtain weights corresponding to various sub-data in the preprocessed data; determine a first risk score based on the preprocessed data and each of the weights; and obtain a time series of the first risk score based on the first risk score corresponding to each of the periods.

[0080] In this embodiment of the disclosure, textual descriptions of the things represented by the first multimodal data and annotations of people and things in the images (videos) of the first multimodal data can be extracted from the first multimodal data, i.e., image annotation. As mentioned above, the first multimodal data may include data of multiple categories. For example, the first multimodal data may include: food images, operating room videos, public opinion data, environmental parameters, and on-site feedback information. After preprocessing, each category of data can correspond to a sub-data. After preprocessing, each category of data in the first multimodal data can correspond to a sub-data of its own category.

[0081] For example, fixed weights can be pre-set for various types of sub-data. For instance, fixed weights can be set according to regulatory focus, or different fixed weights can be set for different types of merchants.

[0082] For example, various types of sub-data can be input into the transformer model to obtain the weights corresponding to each type of sub-data. This allows for dynamic adjustment of the weights corresponding to each type of sub-data based on the first collection of multimodal data each time.

[0083] For example, various types of sub-data can be input into a recurrent neural network model, such as a gated recurrent unit (GRU), to obtain the weights corresponding to each type of sub-data. In this way, the weights corresponding to each type of sub-data can also be dynamically adjusted.

[0084] In this embodiment, preprocessed data and the weights corresponding to each sub-data in the preprocessed data can be input into a large language model to obtain a first risk score. The first multimodal data for each period can be matched with the preprocessed data for that period, thereby obtaining the first risk score for that period. Alternatively, the first risk score and its confidence level can be obtained. Based on the first risk scores for each period, a time series of the first risk scores can be obtained.

[0085] For example, preprocessed data and the weights corresponding to the sub-data in the preprocessed data can be input into a classification model, and processed by the normalization function of the output layer of the classification model to obtain the first risk score.

[0086] For example, the preprocessed data corresponding to each period and the weights corresponding to the sub-data in each preprocessed data can be input into a long short-term memory network to obtain the first risk score time series.

[0087] For example, the preprocessed data corresponding to each period and the weights corresponding to the sub-data in each preprocessed data can be input into the autoregressive integrated moving average model to obtain the first risk score time series.

[0088] The steps performed by the risk assessment unit in this embodiment can be completed by an intelligent analysis platform or by a portable detection terminal.

[0089] In this embodiment, by preprocessing the first multimodal data to remove irrelevant and redundant data, the interference of irrelevant data on determining the first risk score is reduced, thus also reducing the amount of data processed. Furthermore, by combining the weights of each sub-data, the processing focus is clarified, making the first risk score more objective and accurate while better meeting the needs of the scenario. Therefore, the accuracy and scenario adaptability of the first risk score are improved.

[0090] In one possible implementation, the risk assessment unit is further configured to: dynamically determine an alarm threshold based on food safety influencing factors, wherein the food safety influencing factors include at least seasonal factors; compare the first risk score time series with the alarm threshold to obtain a comparison result; and issue an alarm message if the comparison result meets the alarm conditions.

[0091] Food safety influencing factors are both objective and subjective factors that affect the food safety assessment of regulated businesses. For example, food safety influencing factors may include one or more of the following: weather factors, regional factors, and the regulated business's enforcement record.

[0092] For example: Summer is a peak season for food safety issues, so the alarm threshold can be appropriately lowered. Winter is a peak season for food safety issues, so the alarm threshold can be appropriately raised. For areas with good food safety records, the alarm threshold can be appropriately raised. For regulated businesses with multiple enforcement records, the alarm threshold can be appropriately raised. For regulated businesses with no enforcement records and not newly established, the alarm threshold can be appropriately raised.

[0093] In another example, a first neural network can be trained based on reinforcement learning, with seasonal factors and / or regional factors and / or law enforcement records input into the first neural network to obtain an alarm threshold.

[0094] In this embodiment of the disclosure, the values ​​in the first risk score time series can be compared with an alarm threshold. The comparison result can be a numerical comparison result. The alarm condition can represent the condition that triggers the issuance of an alarm message.

[0095] For example, the alarm condition can be: the comparison result indicates that at least one value is higher than the alarm threshold.

[0096] For example, the alarm condition can be: the comparison result indicates that there are consecutive, first number of values ​​that are all higher than the alarm threshold.

[0097] The steps performed by the risk assessment unit in this embodiment can be completed by an intelligent analysis platform or by a portable detection terminal.

[0098] In this embodiment of the disclosure, the alarm threshold can be dynamically adjusted, and the first risk score time series is compared with the alarm threshold to determine whether to issue an alarm message, thereby improving the flexibility and accuracy of the alarm.

[0099] In one possible implementation, the intelligent analysis platform is further configured to: generate a geographic heat map and an optimized route starting from the departure point based on the geographic location of each of the merchants to be subject to on-site enforcement and the time series of the first risk score corresponding to each of the merchants to be subject to on-site enforcement; and send the geographic heat map and the optimized route.

[0100] A geographic heatmap can represent the geographical location and food safety risks of businesses awaiting on-site enforcement. The heatmap can use different colors or different saturations of the same color to represent food safety risks. The starting point can be a preset geographical location, such as a food safety supervision center, or any geographical location of the user. The optimized route can be an enforcement route derived by comprehensively considering the food safety risks of businesses awaiting on-site enforcement and the geographical relationships between them.

[0101] For example, a large language model can be obtained. This large language model has a built-in electronic map containing the geographical location, name, or ID of each merchant to be subject to on-site enforcement. The name or ID of the merchant to be subject to on-site enforcement, along with its corresponding first risk score time series, is input into the large language model to obtain a geographic heatmap. This geographic heatmap can contain 100... 100 grids, or 50 50 grids. This example does not limit the size of the geographic heatmap; the size can be defined according to actual needs or computing power.

[0102] In this embodiment, the geographical locations of each merchant awaiting on-site enforcement are obtained based on a geographic heatmap. Based on these geographical locations and the time series of the first risk scores for each merchant, a weighted calculation is performed to determine the enforcement priority for each merchant. An optimized route is then determined according to the enforcement priority.

[0103] For example, the A-Star Search Algorithm can be used to generate an optimized route map based on the geographical location of the merchant to be subject to on-site enforcement and the time series of the first risk score.

[0104] The intelligent analysis platform can send geographic heat maps and optimized routes, allowing users to intuitively see the relative locations of businesses to be subject to on-site enforcement, as well as the optimized routes obtained through intelligent calculations, for user reference, thereby improving the rationality of user enforcement.

[0105] In one possible implementation, the risk assessment unit is further configured to: extract keywords reflecting food safety issues from text data; identify food in image data to obtain food location markers, and identify food appearance features in image data to obtain food appearance description text; and identify individuals in video data to obtain individual location markers, and identify the illegal actions of each individual to obtain illegal action description text containing the time of the illegal action.

[0106] For food images, text describing the appearance of the food in the image can be generated. For images in on-site feedback information, text describing the operators and operating environment in the image can be generated. For food images, a first bounding box can be used to select the food to indicate its location. In this embodiment of the disclosure, the food location annotation can be a first bounding box.

[0107] Descriptive text depicting inappropriate actions by individuals in the video can be generated from the video data. A second bounding box can be used to select individuals within the video data to indicate their position. In this embodiment, the position marker for individuals can be a second bounding box.

[0108] Keywords reflecting video safety issues can be extracted from text in public opinion data and on-site feedback. These keywords can include those related to serious food safety incidents, video safety complaints, and comments about regulated businesses on social media.

[0109] The steps performed by the risk assessment unit in this embodiment can be completed by an intelligent analysis platform or by a portable detection terminal.

[0110] In this embodiment of the disclosure, different preprocessing can be performed according to the type of the first multimodal data, so that the preprocessed data is easier for the model to process and understand, and also removes redundant or irrelevant data, thereby improving the efficiency and accuracy of determining the first risk score time series.

[0111] In one possible implementation, the risk assessment unit is further configured to: establish a cross-modal correlation graph, the cross-modal correlation graph comprising a central node and edge nodes connected to the central node, the central node representing the geographical location of the regulated merchant, the edge nodes representing sub-data in the preprocessed data, and the edges between the edge nodes and the central node carrying weights corresponding to the sub-data; input the cross-modal correlation graph into a large language model to obtain the first risk score.

[0112] In this embodiment, a cross-modal association graph is used to establish connections between various types of sub-data based on the association between each type of sub-data and the regulated merchant. This presents the various types of sub-data in a structured manner, which not only facilitates the identification of various types of sub-data of the same regulated merchant by the large language model, but also facilitates the discovery of deep and implicit relationships between various types of sub-data, thereby improving the accuracy of the first risk score.

[0113] In one possible implementation, the portable detection terminal is used to add noise to the first multimodal data and the rapid detection image to obtain encrypted data, and then send the encrypted data; the intelligent analysis platform is used to record the encrypted data in the form of a blockchain; the intelligent analysis platform is used to obtain an initial rapid detection result based on the rapid detection image; the intelligent analysis platform is used to process the initial rapid detection result based on zero-knowledge proof technology to obtain a rapid detection result, and then send the rapid detection result, wherein the risk score is used as a law enforcement reference; the portable detection terminal is used to receive and display the geographic heat map, the optimized route, and the alarm information.

[0114] The portable detection terminal can encrypt the collected data (rapid detection images, first multimodal data, etc.). For example, differential privacy technology can be used for encryption to obtain and send encrypted data. The intelligent analysis platform records the hash value of the encrypted data in blockchain form to protect the data security of the monitored merchants. The rapid detection network can be pre-trained. The intelligent analysis platform inputs the rapid detection image into the rapid detection network to obtain the initial rapid detection result. The intelligent analysis platform can use zero-knowledge proof technology to process the initial rapid detection result to obtain a rapid detection result that does not affect use but does not disclose the accurate value of the initial rapid detection result. For example, if the initial rapid detection result is a nitrite content of A mg / L, after processing with zero-knowledge proof technology, the obtained rapid detection result is a nitrite content of less than B mg / L, where A and B are both positive numbers, and A is less than B, and B is a value determined according to food safety standards. In addition, in the embodiments of this disclosure, federated learning can be used to update model parameters without sharing the original data, thus protecting data security. The model here can include the network model involved in the embodiments of this disclosure. Examples include: Transformer model, gated recurrent unit, large language model, classification model, long short-term memory network, first neural network, fast detection network, and autoregressive integrated moving average model.

[0115] The intelligent analysis platform can also send the confidence level of the first risk score and the chain of evidence determining the first risk score to the portable testing terminal. The portable testing terminal can receive and display geographic heat maps, optimized routes, and alarm information. It can also receive and display confidence levels, chains of evidence, and rapid test results, providing users with rich and reliable reference data.

[0116] In this embodiment, the portable detection terminal and the intelligent analysis platform work together to protect data security and improve data safety at the data transmission, data processing, and data storage stages. Furthermore, the portable detection terminal displays multiple processing results, enhancing the accuracy and reliability of law enforcement.

[0117] In one possible implementation, the portable detection terminal performs the second analysis step based on MobileNetV2 and the first analysis step based on a gated loop unit.

[0118] In this embodiment, the portable detection terminal is equipped with a lightweight neural network, which can reduce the computational resource burden on the portable detection terminal while performing the analysis steps, thus balancing computational efficiency and accuracy.

[0119] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the functions of the above-described system.

[0120] This disclosure also provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the functions of the above-described system.

[0121] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program is executed by a processor to implement the functions of the above-described system.

[0122] Figure 4 This is a schematic diagram of the structure of an electronic device for controlling a food safety enforcement system, provided as an embodiment of this disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0123] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TMMac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0124] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to implement the functions of the system described above.

[0125] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0126] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0127] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0128] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0129] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0130] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0132] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A food inspection enforcement system, comprising: The system comprises a data acquisition unit and a risk assessment unit; The data acquisition unit is configured to acquire first multi-modal data of a supervised merchant, the first multi-modal data comprising at least food images representing food appearance, operation room videos representing scenes in a food processing site, and public opinion data representing consumer feedback information; The risk assessment unit is configured to perform a first analysis step, a fast inspection image acquisition step, and a second analysis step; The first analysis step comprises: determining a first risk score time series of the supervised merchant in a future time period based on first multi-modal data acquired in multiple periods; determining whether to issue an alarm information based on the first risk score time series; The fast inspection image acquisition step comprises: acquiring a fast inspection image of a detection tool in the on-site law enforcement merchant indicated in the alarm information; The second analysis step comprises: determining a risk score based on the first multi-modal data and the fast inspection image, the risk score representing a food risk level.

2. The system of claim 1, wherein, The risk assessment unit comprises a portable detection terminal; The portable detection terminal is configured to perform the first analysis step, the fast inspection image acquisition step, and the second analysis step; Alternatively, The risk assessment unit comprises a portable detection terminal and a smart analysis platform; The smart analysis platform is configured to perform the first analysis step and the second analysis step; The portable detection terminal is configured to perform the fast inspection image acquisition step.

3. The system of claim 1, wherein, The risk assessment unit is further configured to: preprocess the first multi-modal data to obtain preprocessed data, the preprocessed data comprising text descriptions and image annotations corresponding to the first multi-modal data; acquire weights corresponding to various types of sub-data in the preprocessed data; determine a first risk score based on the preprocessed data and the weights, and obtain the first risk score time series based on the first risk scores corresponding to the periods.

4. The system of claim 2, wherein, The risk assessment unit is further configured to: dynamically determine an alarm threshold based on food safety influencing factors, the food safety influencing factors comprising at least seasonal factors; compare the first risk score time series with the alarm threshold to obtain a comparison result, and issue an alarm information if the comparison result meets an alarm condition.

5. The system of claim 2, wherein, The smart analysis platform is further configured to: generate a geographic heat map and an optimized route with a departure location as a starting point according to geographic locations of the on-site law enforcement merchants and the first risk score time series corresponding to the on-site law enforcement merchants; and send the geographic heat map and the optimized route.

6. The system of claim 1, wherein, The first multi-modal data further comprises environmental parameters and on-site feedback information of the supervised merchant.

7. The system of claim 3, wherein, The risk assessment unit is further configured to: extract keywords reflecting food safety issues from text data; identify food in image data to obtain food location annotations, and identify food appearance features in the image data to obtain food appearance description texts; identify characters in video data to obtain character location annotations, and identify irregular actions of the characters to obtain irregular action description texts containing time points.

8. The system of claim 3, wherein, The risk assessment unit is further configured to: establish a cross-modal correlation graph, the cross-modal correlation graph comprising a center node and an edge node connected to the center node, the center node representing a geographic location of a supervised merchant, the edge node representing a sub-data in the pre-processed data, and the edge between the center node and the edge node carrying a weight corresponding to the sub-data; input the cross-modal correlation graph into a large language model to obtain the first risk score.

9. The system of claim 5, wherein the portable detection terminal is configured to add noise to the first multi-modal data and the quick detection image to obtain encrypted data, and send the encrypted data; the intelligent analysis platform is configured to record the encrypted data in a blockchain form; the intelligent analysis platform is configured to obtain an initial quick detection result based on the quick detection image; the intelligent analysis platform is configured to process the initial quick detection result based on zero-knowledge proof technology to obtain a quick detection result, and send the quick detection result, the risk score serving as a law enforcement reference; the portable detection terminal is configured to receive and display the geographic heat map, the optimized route, and the warning information.

10. The system of claim 2, wherein, The portable detection terminal executes the second analysis step based on MobileNetV2 and executes the first analysis step based on a gated recurrent unit.

Images