Fresh fruit cold chain monitoring system based on edge calculation

Abstract

The invention discloses a fresh fruit cold chain monitoring system based on edge calculation. The fresh fruit cold chain monitoring system comprises an image acquisition module which acquires fresh fruit pictures in a carriage through a camera; the multi-source sensor array is used for acquiring real-time data through various types of sensors; the edge computing device is used for identifying the maturity corresponding to the fresh fruits according to the acquired image data, and judging whether the environment in the carriage is within the optimal environment threshold value of the maturity or not by adopting a lightweight convolutional network in combination with the environment data in the carriage acquired by the sensor; and the server application is used for transmitting the sensor data, the maturity information and the regulation and control decision data to the cloud platform for storage under the condition that a signal transmission condition is met, and sending the information to a user for reminding. The invention provides a fresh fruit cold chain monitoring system based on edge calculation, which can adjust the environment in the transportation process in time according to the maturity change of fresh fruits in the transportation process.

Description

Technical Field

[0001] This invention relates to the field of cold chain transportation technology, and in particular to a fresh fruit cold chain monitoring system based on edge computing. Background Technology

[0002] Currently, cold chain transportation is required for the transportation of fresh fruit. This means that the temperature and humidity inside the vehicle must be controlled to meet the storage conditions of different fresh fruits, thereby preventing the transported goods from spoiling.

[0003] In the current cold chain transportation process, temperature control is mainly achieved by the driver setting the corresponding compartment temperature according to transportation needs. However, real-time adjustments cannot be made throughout the transportation process. Furthermore, the compartment environment is constantly changing due to the influence of the external environment. This leads to deviations in the optimal storage environment caused by changes in the ripeness of fresh fruit during long-distance and long-term transportation. Therefore, it is necessary to adjust the temperature according to the condition of the fresh fruit in the compartment in a timely manner to better prevent the transported goods from spoiling. However, the primary responsibility of current cold chain transportation drivers is to drive the vehicle, and they do not have the relevant knowledge to adjust the compartment environment according to changes in the transportation conditions of different fresh fruits. Summary of the Invention

[0004] The purpose of this invention is to provide a fresh fruit cold chain monitoring system based on edge computing, which adjusts the environment in a timely manner according to changes in the ripeness of fresh fruit during transportation.

[0005] The technical solution adopted by the edge computing-based fresh fruit cold chain monitoring system disclosed in this invention is as follows: A fresh fruit cold chain monitoring system based on edge computing includes: Image acquisition module: Captures images of fresh fruit inside the truck using a camera, and performs image acquisition at set time intervals during transportation; Multi-source sensor array: Real-time data collected by various types of sensors; Edge computing device: The collected image data identifies the ripeness of the fresh fruit. Combined with the environmental data inside the carriage collected by the sensors, a lightweight convolutional network is used to determine whether the environment inside the carriage is within the optimal environmental threshold for the ripeness. If it exceeds the threshold, the environmental conditions inside the carriage are adjusted in time through the onboard equipment. If it does not exceed the threshold, the current conditions inside the carriage are maintained. Intelligent environmental control: Based on maturity recognition and environmental comparison results, and according to information in the edge computing device database, the temperature, humidity and gas concentration inside the cold chain vehicle are automatically adjusted to meet the preservation needs of fresh fruits at different maturity levels. Server-side application: When signal transmission conditions are available, sensor data, maturity information, and control decision data are transmitted to the cloud platform for storage, and information is sent to users for reminders.

[0006] As a preferred embodiment, the camera is equipped with autofocus and exposure adjustment to adapt to different lighting conditions for capturing RGB images of fresh fruit, and the vehicle compartment is equipped with soft lighting to improve recognition accuracy.

[0007] As a preferred embodiment, the sensors include temperature and humidity, oxygen, carbon dioxide and ethylene sensors. The sensor data is acquired by an STM32 series microcontroller unit via serial communication UART and IIC, and then transmitted to the edge computing device via serial communication UART.

[0008] As a preferred option, one of the lightweight convolutional neural networks SqueezeNet, Xception, ShuffleNet, and MobileNet is selected, and further lightweight processing is performed through TensorRT framework acceleration and INT8 model quantization to convert the model into ONNX format, realizing a real-time processing function model on edge computing devices, which can make carriage adjustment decisions under no-signal conditions.

[0009] As a preferred embodiment, the model uses lightweight convolutional neural network operations to extract features such as color, shape, and texture of fruit in RGB images. To balance performance and speed, depthwise separable convolution is used instead of standard convolution, decomposing standard convolution into two steps: depthwise convolution and pointwise convolution. The formulas for depthwise convolution and pointwise convolution are as follows: in, For the input feature map, To output the feature map, For convolution kernel, For the number of channels, For channel indexing, These are the position coordinates; Based on these features, the Softmax activation function is used to map the features to class probabilities, thereby identifying the fruit type and classifying its ripeness. The following is the formula for the Softmax function. in, It is the score of the i-th category, which is the score for each fruit variety and its ripeness; Considering that the ripeness of multiple different individual fruits will be tested, the decision will be made based on the percentage of fruits with the highest ripeness. The database in the edge computing device stores information such as the freezing point of different fruits at different stages of ripeness and the range of storage gas concentrations, which will be used to regulate the environment in the train carriage.

[0010] As a preferred embodiment, when the concentrations of gases such as oxygen, carbon dioxide, and ethylene exceed a threshold, the ventilation equipment will be activated to circulate external gas and adjust it to within the threshold range; when the temperature exceeds a threshold, the refrigeration equipment will be adjusted up or down to bring the temperature close to the freezing point of the current maturity level; when the humidity is below a threshold, the humidifier will be activated to increase the humidity in the vehicle compartment, etc. The threshold processing formula is as follows: in, This indicates that the corresponding control equipment has been activated. Indicates whether to close or maintain the current state. Indicates the first Each sensor measurement value, This indicates the set threshold.

[0011] As a preferred solution, users can monitor transportation environment parameters and fruit ripeness status in real time through server-side applications. Edge computing devices upload the data collected by sensors and the decision data made to the server platform via Narrowband Internet of Things (NB-IoT) and MQTT protocols for subsequent queries and user interaction.

[0012] The beneficial effects of the edge computing-based fresh fruit cold chain monitoring system disclosed in this invention are as follows: It utilizes cameras to capture images of the fresh fruit inside the truck, and combines this with real-time monitoring of the environment within the truck using various types of sensors. A lightweight convolutional neural network model is used to accurately identify the fruit's ripeness and to fuse and analyze data from multiple sensor types, thereby promptly determining the state of the fresh fruit inside the truck and adjusting the environment accordingly. Furthermore, by mounting edge computing devices on the transport vehicle, reliance on centralized cloud servers is reduced, ensuring intelligent decision-making even in areas without signal coverage, thus achieving timely adjustments. This eliminates the need for analysis and judgment by other technical personnel, preventing situations where drivers lack the necessary knowledge and are unable to operate the system. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the structure of a fresh fruit cold chain monitoring system based on edge computing according to the present invention.

[0014] Figure 2 This is a flowchart of a fresh fruit cold chain monitoring system based on edge computing according to the present invention. Detailed Implementation

[0015] The present invention will be further described and illustrated below with reference to specific embodiments and the accompanying drawings: Please refer to Figure 1 A fresh fruit cold chain monitoring system based on edge computing includes: Image acquisition module: The camera captures images of fresh fruit inside the truck bed and takes pictures at set time intervals during transportation. The camera has automatic focus and exposure adjustment to adapt to different lighting conditions and captures RGB images of the fresh fruit. The truck bed is also equipped with soft lighting to avoid bright areas on the surface of the fresh fruit caused by strong light in the dark environment of the truck bed, thus improving recognition accuracy.

[0016] Furthermore, considering that fresh fruit is stacked in the compartment during actual cold chain transportation, which is not conducive to image data collection, a batch of fresh fruit with its outer packaging removed can be placed directly in the compartment environment to facilitate data collection.

[0017] Multi-source sensor array: Real-time data collected by various types of sensors, including temperature, humidity, oxygen, carbon dioxide and ethylene sensors. Since the fresh fruit is stacked, the sensors can be easily placed in multiple corners of the carriage, or some can be placed between the fresh fruit during the stacking process to detect parameters at various locations in the carriage and provide detection accuracy.

[0018] Furthermore, sensor data is acquired via serial communication UART and IIC through the STM32 series microcontroller unit, and then transmitted to the edge computing device via serial communication UART.

[0019] Edge computing devices: The collected image data is used to identify the ripeness of the fresh fruit. Combined with the environmental data inside the carriage collected by the sensors, a lightweight convolutional network is used to determine whether the environment inside the carriage is within the optimal environmental threshold for that ripeness.

[0020] Edge computing devices are equipped with high-performance GPUs, CPUs, and neural network acceleration units to process data from cameras and sensors. They should have more than 16GB of RAM and more than 128GB of hard drive space to meet the computational and data storage requirements of the models.

[0021] One of the lightweight convolutional neural networks SqueezeNet, Xception, ShuffleNet, and MobileNet is selected, and further lightweight processing is carried out through TensorRT framework acceleration and INT8 model quantization. The model is then converted into ONNX format to realize real-time processing function model on edge computing device, which can make carriage adjustment decisions under no-signal conditions.

[0022] In actual transportation, different fruits have different sizes, shapes, colors, and other characteristics, which are used to distinguish fruit varieties. Different ripening stages of the same fruit also exhibit different textures and color proportions, which are used to classify the different ripening stages. For example, apples, plums, and grapes have significant differences in shape, size, and color, which can be used to distinguish them; ripeness can also be used for classification.

[0023] For example, the ripeness of white apricots can be divided according to the proportion of green in the fruit. In the unripe stage, the green content of the fruit is 30% to 50%; in the semi-ripe stage, the green content is 10% to 30%; and in the fully ripe stage, the green content is less than 10%.

[0024] The model uses lightweight convolutional neural network operations to extract features such as color, shape, and texture of fruit in RGB images. To balance performance and speed, depthwise separable convolution is used instead of standard convolution, decomposing standard convolution into two steps: depthwise convolution and pointwise convolution. The formulas for depthwise convolution and pointwise convolution are as follows: in, For the input feature map, To output the feature map, For convolution kernel, For the number of channels, For channel indexing, These are the position coordinates; Based on these features, the Softmax activation function is used to map the features to class probabilities, thereby identifying the fruit type and classifying its ripeness. The following is the formula for the Softmax function. in, It is the score of the i-th category, which is the score for each fruit variety and its ripeness; Considering that the ripeness of multiple different individual fruits will be tested, the decision will be made based on the percentage of fruits with the highest ripeness. The database in the edge computing device stores information such as the freezing point and storage gas concentration range of different fruits at different stages of ripeness, which will be used to regulate the environment of the vehicle compartment. For example, the freezing point of winter jujubes is -3℃ when they are white-ripe and -6℃ when they are fully red, while the near-freezing temperature should be maintained between 0.3℃ and 0.8℃ above the freezing point.

[0025] Intelligent environmental control: Based on maturity recognition and environmental comparison results, and according to information in the edge computing device database, the temperature, humidity and gas concentration inside the cold chain vehicle are automatically adjusted to meet the preservation needs of fresh fruits at different maturity levels.

[0026] When the concentrations of gases such as oxygen, carbon dioxide, and ethylene exceed the threshold, the ventilation equipment will be activated to circulate external air and adjust it to within the threshold range; when the temperature exceeds the threshold, the refrigeration equipment will be adjusted up or down to bring the temperature close to the freezing point of the current maturity level; when the humidity is below the threshold, the humidifier will be activated to increase the humidity in the vehicle compartment, etc. The following is the threshold handling formula: in, This indicates that the corresponding control equipment has been activated. Indicates whether to close or maintain the current state. Indicates the first Each sensor measurement value, This indicates the set threshold.

[0027] Server-side application: When signal transmission conditions are available, sensor data, maturity information, and control decision data are transmitted to the cloud platform for storage, and information is sent to users for reminders. Users can monitor transportation environment parameters and fresh fruit maturity status in real time through the server-side application. Edge computing devices upload the data collected by sensors and the decision data made through narrowband IoT and MQTT protocols to the server-side platform for subsequent querying and user interaction.

[0028] The vehicle can also be equipped with a touch screen, which can be used to connect with edge computing devices, allowing staff to view data information locally.

[0029] Please refer to Figure 2 The entire monitoring process is as follows: At the start of each transportation cycle, cameras capture images of fresh fruit inside the truck, and a neural network model deployed in edge computing devices identifies the ripeness level. Simultaneously, by comparing real-time data collected from various types of sensors, it determines whether the environment inside the truck is within the optimal threshold for that ripeness level. If the threshold is exceeded, the environment inside the truck is automatically adjusted, such as through fans, humidifiers, and temperature control. Under signal conditions, sensor data, ripeness information, and control decision data are transmitted to a cloud platform for storage, and an alert is sent to the user via SMS. If the threshold is not exceeded, the next cycle begins.

[0030] The above information can also be displayed on a touch screen connected to the edge computing device.

[0031] This invention provides a fresh fruit cold chain monitoring system based on edge computing. It utilizes cameras to capture images of the fresh fruit inside the truck compartment and combines this with various types of sensors to monitor the environment in real time. A lightweight convolutional neural network model is used to accurately identify the fruit's ripeness and to fuse and analyze data from multiple sensor types, thereby promptly determining the state of the fresh fruit inside the compartment and making timely adjustments to the environment. Furthermore, by mounting edge computing devices on the transport vehicle, reliance on centralized cloud servers is reduced, ensuring intelligent decision-making even in areas without signal coverage. This allows for timely adjustments without requiring additional technical personnel for analysis and judgment, preventing situations where drivers lack the necessary knowledge and are unable to operate the system.

[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A fresh fruit cold chain monitoring system based on edge computing, characterized by, Comprise: Image acquisition module: through the camera to collect fresh fruit pictures in the carriage, and in the transportation process, according to the set time interval for collection; Multi-source sensor array: through the real-time data collected by multiple types of sensors; Edge computing device: the collected image data is identified as the maturity of fresh fruit, combined with the environmental data collected by the sensor, the light convolutional network is used to judge whether the environment in the carriage is within the best environmental threshold of the maturity, if it exceeds the threshold, the device carried is used to adjust the environmental conditions in the carriage in time, if it does not exceed the threshold, the current carriage conditions are maintained; Intelligent environment regulation: based on the maturity recognition and environmental comparison results, according to the information in the database of edge computing device, the temperature, humidity and gas concentration in the cold chain vehicle are automatically adjusted to meet the preservation needs of fresh fruits of different maturity; Server application: under the condition of signal transmission, the sensor data, maturity information and regulation decision data are transmitted to the cloud platform for storage, and the information is sent to the user for reminding.

2. A fresh fruit cold chain monitoring system based on edge computing as claimed in claim 1, wherein, The camera has automatic focusing and exposure adjustment to adapt to different lighting conditions, which is used to shoot RGB images of fresh fruits, and the carriage is equipped with soft light for irradiation to improve the recognition accuracy.

3. A fresh fruit cold chain monitoring system based on edge computing as claimed in claim 1, wherein, The sensor includes temperature and humidity, oxygen, carbon dioxide and ethylene sensors. The sensor data is collected by STM32 series micro control unit through serial communication UART and IIC, and is transmitted to the edge computing device through serial communication UART.

4. A fresh fruit cold chain monitoring system based on edge computing as claimed in claim 3, wherein, Select one of the light convolutional neural network SqueezeNet, Xception, ShuffleNet and MobileNet, and further light weight processing through TensorRT framework acceleration and INT8 model quantization to convert the model to ONNX format, realize the real-time processing function model on the edge computing device, and make the carriage adjustment decision under the condition of no signal.

5. A fresh fruit cold chain monitoring system based on edge computing as claimed in claim 4, wherein, The model uses light convolutional neural network operation to extract the color, shape and texture of the fruit in the RGB image, and uses depth separable convolution instead of standard convolution to balance performance and speed, which decomposes standard convolution into two steps: depth convolution and point convolution. The formulas of depth convolution and point convolution are as follows: wherein, is an input feature map, is an output feature map, is a convolution kernel, is a number of channels, is a channel index, is a position coordinate; According to these characteristics, the Softmax activation function is used to map the features to the category probability, which identifies the fruit type and classifies the maturity at the same time. The formula of Softmax function is as follows: wherein, is the score of the i-th class, i.e. the score of each fruit variety and ripeness. Considering the maturity detection of multiple different fruit individuals, the maturity with the largest proportion is selected for decision; The database in the edge computing device stores the freezing point and storage gas concentration range of different fruits at different maturity, which is used to regulate the carriage environment.

6. A fresh fruit cold chain monitoring system based on edge computing as claimed in claim 3, wherein, When the concentration of oxygen, carbon dioxide, ethylene and other gases exceeds the threshold, the ventilation equipment is started to circulate external gas and adjust to the threshold range; when the temperature exceeds the threshold, the refrigeration equipment is adjusted up or down to adjust the temperature to the near freezing point temperature of the current maturity; when the humidity is lower than the threshold, the humidifier is started to increase the humidity in the carriage, etc. The threshold processing formula is as follows: wherein, indicates to start the respective control device, indicates to switch off or to keep the current state, indicates the measurement value of the sensor, indicates the set threshold value.

7. The fresh fruit cold chain monitoring system based on edge computing as claimed in claim 1, wherein, The user can monitor the transport environment parameters and the fresh fruit ripeness state in real time through the service end application, the edge computing device uploads the data collected by the sensor and the decision data to the service end platform through narrowband Internet of Things (NB-IoT) and MQTT protocol, and then carries out the subsequent query and user interaction.

Images