Process monitoring method and system thereof

Abstract

The invention discloses a process monitoring method and system, and relates to the field of automatic control, and the process monitoring method comprises the steps: calculating a medium temperature according to an ultrasonic transmitting node and an ultrasonic receiving node, and processing first image information to obtain a first behavior feature; performing first early warning judgment according to the medium temperature and the first behavior characteristic; processing the second image information to obtain a second behavior feature; obtaining weight data information, performing weight change judgment according to the weight data information, and performing category matching on the taken raw materials according to a weight change judgment result; performing second early warning judgment according to a matching result of the second behavior characteristic and the taken raw material category; timing judgment is carried out according to the first early warning judgment result, timing is carried out based on the timing judgment result, and third early warning judgment is carried out according to the timing result and the first behavior characteristic. According to the invention, early warning judgment can be carried out on the behavior that raw materials are not put as required in the stir-frying process, so that the real-time supervision on the food stir-frying process is realized.

Description

Technical Field

[0001] The present invention relates to the field of automatic control, and in particular to a process monitoring method and a system thereof. Background Art

[0002] In the process of food frying, such as the frying of hot pot base, the control of oil temperature, the order of raw material addition and the grasp of frying time are all factors that affect product quality. Although the frying equipment currently available on the market has realized the automation of the frying process to a certain extent, and can display the frying process to the processing personnel through the visual window so that they can operate according to the established process, when performing frying tasks, it is inevitable that the timing of raw material addition or the wrong order of addition will occur. These subtle deviations may trigger a chain reaction during the frying process, resulting in the final fried product failing to meet the expected processing requirements, thereby causing a waste of raw materials. This not only increases production costs, but also affects production efficiency and the stability of product quality. Summary of the invention

[0003] Therefore, in order to solve the above-mentioned shortcomings, the present invention is proposed here. Through the present invention, early warning and judgment can be made on the behavior of not adding raw materials as required during the frying process, thereby realizing real-time supervision of the food frying process, timely reminding the operator to correct the operation behavior, reducing the occurrence of improper timing or wrong order of adding raw materials, and reducing the waste of raw materials.

[0004] In one aspect, the present invention provides a process monitoring method, comprising: Obtaining an ultrasonic transmitting node and an ultrasonic receiving node, and calculating a medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node; Acquire first image information, process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain a first behavior feature; Perform a first early warning judgment according to the medium temperature and the first behavior characteristic, obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result; Acquire second image information, process the second image information to obtain a second motion feature, and perform a second behavior feature analysis based on the second motion feature to obtain a second behavior feature; specifically, the second image information is dynamic image information of a raw material placement point; Obtaining weight data information of a preset position, performing weight change judgment according to the weight data information, obtaining a weight change judgment result, performing a category match of the taken-away raw materials according to the weight change judgment result, and obtaining a category match result of the taken-away raw materials; Perform a second early warning judgment based on the second behavior feature and the matching result of the taken-away raw material category, obtain a second early warning judgment result, and generate a second alarm signal based on the second early warning judgment result; Performing a timing judgment according to the first warning judgment result, and performing timing based on the timing judgment result to obtain a timing result; A third early warning judgment is performed according to the timing result and the first behavior feature to obtain a third early warning judgment result, and a third alarm signal is generated based on the third early warning judgment result.

[0005] Furthermore, the calculation of the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node includes: Calculate the ultrasonic flight time according to the ultrasonic transmitting node and the ultrasonic receiving node; The single-point medium temperature is calculated based on the ultrasonic flight time to obtain the single-point medium temperature; The medium temperature is calculated based on the single-point medium temperature to obtain the medium temperature.

[0006] Further, the performing the first behavior feature analysis based on the motion feature to obtain the first behavior feature includes: Extracting a first target feature from the first motion feature, and performing first target feature motion calculation based on the first target feature; A first behavior match is performed according to a result of the first target feature motion calculation to obtain a first behavior feature.

[0007] Further, the performing the first target feature motion calculation based on the first target feature includes: Based on the first target feature, extracting a first target feature point in the first target feature; Calculate the vector of the first target feature point between adjacent frames to obtain a unit first target feature vector; Calculate the first target rotation angle according to the unit first target characteristic vector to obtain the first target rotation angle; The performing first behavior matching according to the result of the first target feature motion calculation to obtain the first behavior feature includes: According to the unit first target feature vector and the first target feature rotation angle, a match is performed with the vector, rotation angle-behavior corresponding data sample set. If a match is made, the first behavior is established, otherwise it is not established.

[0008] Further, the performing a second behavior feature analysis based on the second motion feature to obtain the second behavior feature includes: Extracting a second target feature from the second motion feature, and performing second target feature motion calculation based on the second target feature; A second behavior match is performed based on the result of the second target feature motion calculation to obtain a second behavior feature.

[0009] Furthermore, the performing motion calculation of the second target feature based on the second target feature includes: Based on the second target feature, extracting a second target feature point in the second target feature; Calculate the vector of the second target feature point between adjacent frames to obtain a unit second target feature vector; Calculate the displacement of the second target feature point between adjacent frames to obtain a unit second target displacement; Calculate the second target displacement amount according to the unit second target displacement amount to obtain the second target displacement amount; The performing second behavior matching according to the result of the second target feature motion calculation includes: According to the unit second target characteristic vector and the second target displacement, the vector, displacement-behavior corresponding data sample set is matched. If they match, the second behavior is established, otherwise it is not established.

[0010] On the other hand, the present invention further provides a process monitoring system, which is used to execute the process monitoring method, and the process monitoring system includes: A data acquisition unit, used to obtain an ultrasonic transmitting node, an ultrasonic receiving node, first image information, second image information, and weight data information; A temperature calculation unit, used for calculating the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node; A first behavior analysis unit is used to process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain the first behavior feature; A first early warning judgment unit, used to perform a first early warning judgment according to the medium temperature and the first behavior characteristic, obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result; A second behavior analysis unit is used to obtain second image information, process the second image information to obtain a second motion feature, and perform a second behavior feature analysis based on the second motion feature to obtain a second behavior feature; A raw material type judgment unit is used to obtain weight data information of a preset position, perform weight change judgment according to the weight data information, obtain a weight change judgment result, perform a category match of the raw materials to be removed according to the weight change judgment result, and obtain a category match result of the raw materials to be removed; A second early warning judgment unit is used to perform a second early warning judgment according to the second behavior feature and the matching result of the category of the taken raw materials, obtain a second early warning judgment result, and generate a second alarm signal based on the second early warning judgment result; A timing unit, performing a timing judgment according to the first warning judgment result, performing timing based on the timing judgment result, and obtaining a timing result; The third early warning judgment unit is used to perform a third early warning judgment according to the timing result and the first behavior feature, obtain a third early warning judgment result, and generate a third alarm signal based on the third early warning judgment result.

[0011] The present invention has the following advantages: The present invention monitors the medium temperature, the staff's material taking and material putting behaviors, the types of raw materials taken away and the frying time, and makes early warning judgments on the behavior of not putting raw materials according to requirements during the frying process, thereby realizing real-time supervision of the food frying process, timely reminding the operator to correct the operation behavior, reducing the occurrence of improper raw material putting timing or wrong putting sequence, reducing the waste of raw materials, and improving production efficiency and product quality stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of the process monitoring method; Figure 2 It is a logical structure diagram of the process monitoring system; Figure 3 yes Figure 2 A schematic diagram of the logical structure of a process monitoring module in the process monitoring system shown; 100. Ultrasonic transmitting module; 200. Ultrasonic receiving module; 300. Image acquisition module; 400, weight sensor module; 500. Remote monitoring platform; 600, process monitoring module; 610, data acquisition unit; 620, temperature calculation unit; 630, first behavior analysis unit; 640, timing unit; 650, second behavior analysis unit; 660, raw material type judgment unit; 670, first early warning judgment unit; 680, third early warning judgment unit; 690, second early warning judgment unit; 700. Alarm module. DETAILED DESCRIPTION

[0013] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0014] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0015] As described in the background technology, the frying equipment currently available on the market has realized the automation of the frying process to a certain extent, and can display the frying process to the processing personnel through a visual window so that they can operate according to the established process. However, when performing the frying task, it is inevitable that the raw materials are put in at an inappropriate time or in the wrong order. These slight deviations may trigger a chain reaction during the frying process, resulting in the final fried product failing to meet the expected processing requirements, thereby causing a waste of raw materials. This not only increases production costs, but also affects production efficiency and the stability of product quality.

[0016] Embodiment 1: Therefore, in order to solve the above technical problems existing in the prior art, a process monitoring method is proposed herein, which includes: S100: Acquire an ultrasonic transmitting node and an ultrasonic receiving node, and calculate a medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node; Specifically, the calculation of the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node includes: S110: Calculate the ultrasonic flight time according to the ultrasonic transmitting node and the ultrasonic receiving node. In this embodiment, the ultrasonic generator and the ultrasonic receiver are arranged corresponding to each other, and at least one group of ultrasonic generator and ultrasonic receiver is arranged. The specific calculation method of the ultrasonic flight time is as follows: ; in, is the ultrasonic flight time, in seconds; is the ultrasonic emission node, in seconds; is the ultrasonic receiving node, in seconds; n is the number of ultrasonic transmitters and the corresponding ultrasonic receivers.

[0017] S120: Calculate the single-point medium temperature based on the ultrasonic wave flight time to obtain the single-point medium temperature; Specifically, the single-point medium temperature is calculated as follows: ; in, For the The single-point medium temperature at the location of the ultrasonic transmitter and the ultrasonic receiver, in degrees Celsius; For the The distance between the ultrasonic transmitter and the ultrasonic receiver, in meters; The speed of sound waves in a medium at 0 degrees Celsius, in meters per second; is the change in the speed of sound waves when the temperature of the medium changes, in meters per second per degree Celsius; , All are determined by the properties of the medium; S130: Calculate the medium temperature according to the single-point medium temperature to obtain the medium temperature; Specifically, the medium temperature is calculated as follows: ; in, is the medium temperature in degrees Celsius.

[0018] S200: Acquire first image information, process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain a first behavior feature; Specifically, the first image information can be obtained by shooting an image of a first preset position through an image capture device, such as a camera, a video camera, etc. In this embodiment, the first preset position is the position of the wok, and the first image information is dynamic image information of the position of the wok. In this embodiment, the first image information is processed by an image recognition algorithm (such as a motion target detection algorithm based on RLBSP texture and HSV), including background model initialization, foreground detection, pixel classification, separation of foreground and background, etc., so as to separate the motion features in the first image. In this embodiment, the first motion feature is a container containing raw materials.

[0019] In this embodiment, the step of performing a first behavior feature analysis based on the first motion feature to obtain the first behavior feature includes: S210: extracting a first target feature from the first motion feature, and performing first target feature motion calculation based on the first target feature; Specifically, the specific calculation method of the first target feature motion calculation is as follows: S211: extracting a first target feature point in the first target feature based on the first target feature; S212: Calculate the vector of the first target feature point between adjacent frames to obtain a unit first target feature vector. The specific calculation method is as follows: ; in,m is the number of frames, is the unit first target feature vector; , For the The horizontal and vertical coordinates of the first target feature point in the first image of the frame, , For the m The horizontal and vertical coordinates of the first target feature point in the first image of the frame.

[0020] S213: Calculate the first target rotation angle according to the unit first target feature vector to obtain the first target rotation angle. The calculation method is as follows: ; in, is the first target rotation angle, C is the number of the first target feature points.

[0021] S220: Perform first behavior matching according to the result of the first target feature motion calculation to obtain the first behavior feature. Specifically, match the first target feature vector and the first target feature rotation angle with the vector, rotation angle-behavior corresponding data sample set. If they match, the first behavior is established, otherwise it is not established.

[0022] S300: performing a first early warning judgment according to the medium temperature and the first behavior characteristic, obtaining a first early warning judgment result, and generating a first alarm signal according to the first early warning judgment result; Specifically, when the first behavior is established, if the medium temperature is lower or higher than the preset temperature range, it is judged as "the operation is not performed as required", and an alarm signal is generated; when the first behavior is not established, if the medium temperature is higher than the preset temperature range, it is also judged as "the operation is not performed as required", and a first alarm signal is generated; the first alarm signal is output to the alarm module so that the alarm module can alarm in at least one of the following ways, including but not limited to sound, light, and text messages. When the first behavior is established, the medium temperature is within the preset temperature range, and it is judged as "the operation is performed as required", and the first alarm signal is not generated; through the above method, the probability of putting raw materials into the frying equipment when the oil temperature does not reach the preset temperature or the temperature is higher than the preset temperature can be reduced.

[0023] Exemplarily, taking the preparation of hot pot base as an example, before frying, edible oil needs to be injected into the wok. After the edible oil is heated to a preset temperature, the raw materials will be put in order. In this process, the temperature of the edible oil is measured by at least one group of ultrasonic transmitting modules and ultrasonic receiving modules arranged on the side of the wok. The specific measurement method can refer to steps S110 to S130 to obtain the current temperature of the edible oil. At the same time, the image of the wok position area is collected through an image acquisition module (such as a camera). When the first motion feature is identified in the image (that is, the container containing the raw materials), the rotation angle of the container is monitored and calculated in real time through steps S211 to S213. When the rotation angle of the container reaches a certain angle, it is determined that the first behavior (that is, the feeding behavior) is established. If the temperature reaches the preset temperature or the highly preset temperature at this time, a first alarm signal is generated. After the alarm module receives the first alarm signal, it starts to alarm.

[0024] S400: Acquire second image information, process the second image information to obtain a second motion feature, perform a second behavior feature analysis based on the second motion feature, and obtain a second behavior feature; specifically, the second image information is dynamic image information of the raw material placement point. In this embodiment, the second motion feature is a container containing the raw material.

[0025] In this embodiment, the step of performing the second behavior feature analysis based on the motion feature to obtain the second behavior feature includes: S410: extracting a second target feature from the second motion feature, and performing second target feature motion calculation based on the second target feature; Specifically, the specific calculation method of the second target feature motion calculation is as follows: S411: extracting second target feature points in the second target feature based on the second target feature; S412: Calculate the vector of the second target feature point between adjacent frames to obtain a unit second target feature vector. The specific calculation method may refer to the calculation method of the unit first target feature vector. S413: Calculate the displacement of the second target feature point between adjacent frames to obtain the unit second target displacement. The specific calculation method is as follows: ; in, is the unit second target displacement; For the m +1 frame of the horizontal coordinate of the second target feature point in the second image; For the m The horizontal coordinate of the second target feature point in the second image of the frame; S414: Calculate the second target displacement, the specific calculation method is as follows: ; in, W is the second target displacement.

[0026] S420: Perform second behavior matching according to the result of the second target feature motion calculation to obtain the second behavior feature. Specifically, the unit second target feature vector and the second target displacement are matched with the vector, displacement-behavior corresponding data sample set. If they match, the second behavior is established, otherwise it is not established.

[0027] S500: Obtaining weight data information of a preset position, performing weight change judgment according to the weight data information, obtaining a weight change judgment result, performing a category matching of the removed raw materials according to the weight change judgment result, and obtaining a category matching result of the removed raw materials; Specifically, the preset position can be a raw material placement platform, on which a container containing raw materials of a certain weight can be placed. A weight sensing module can be set at the preset position, and the weight sensing module can be bound to the raw material placement platform. The weight data information of the placement platform can be monitored in real time through the weight sensing module. Since each placement platform is used to place specific raw materials, when the container at the preset position is taken away, the weight of the placement platform changes. At this time, the raw material category can be matched according to the number, mark, etc. of the placement platform where the weight change occurs. The number or mark-category data table can be used to match the number or mark of the placement platform where the weight change occurs to match the category of the raw material taken away.

[0028] S600: Perform a second early warning judgment based on the second behavior feature and the matching result of the taken-away raw material category, obtain a second early warning judgment result, and generate a second alarm signal based on the second early warning judgment result.

[0029] Specifically, when the second behavior is established, if the category of the raw materials taken away does not match the raw materials of the current step, a second alarm signal is generated; if the category of the raw materials taken away matches the raw materials of the current step, no second alarm signal is generated; through the above method, when the wrong raw materials are taken, timely reminders can be given, thereby reducing the probability of wrong raw materials.

[0030] Exemplarily, taking the frying of hot pot base as an example, the frying ingredients are usually placed in a specific area or platform. Generally, the ingredients are pre-poured into a container and weighed and then transported to the area or platform. Different ingredients are placed on the corresponding platform, and the image of each area or platform is collected by the image acquisition module. When taking the ingredients, the movement direction and movement displacement of the container are monitored and calculated through steps S411 to S414. When the movement direction meets the preset movement direction and the movement displacement reaches the preset displacement, it is determined that the second behavior is established (that is, the material taking behavior is established). At the same time, by calculating the front and back weights of each platform or area, it is determined whether the weight of each platform or area has changed. The type of raw material is matched according to the number or mark of each platform or area where the weight has changed. For example, the platform where the weight has changed is numbered 2, and the type of raw material corresponding to the platform number 2 in the number-category data table is ginger. At this time, it is determined that the taken raw material is ginger, and the taken raw material is matched with the raw material recorded in the current step. If it does not match, an alarm is issued.

[0031] S700: Performing a timing judgment according to the first warning judgment result, and performing timing based on the timing judgment result to obtain a timing result; Specifically, when the first warning judgment result is "the operation is not performed as required", the timing unit determines that the timing operation cannot be performed. If the first warning judgment result is "the operation is performed as required", the timing unit determines that the timing operation can be performed and performs the timing operation.

[0032] S800: Perform a third warning judgment according to the timing result (the timing result may be the length of time) and the first behavior feature, obtain a third warning judgment result, and generate a third alarm signal based on the third warning judgment result.

[0033] Specifically, when the temperature of the medium reaches the preset temperature, the raw material is put into the medium and the timing is started. When the first action of the next process occurs, if the timing does not reach the preset time, it is judged as "not meeting the requirements" and an alarm signal is generated; after the timing exceeds the preset time, regardless of whether the first action occurs, it is judged as "not meeting the requirements" and an alarm signal is generated; when the first action of the next process occurs, if the timing reaches the preset time, it is judged as "meeting the requirements" and no alarm signal is generated. The above method can reduce the occurrence of insufficient or excessive frying time of raw materials.

[0034] Exemplarily, when the temperature reaches the preset temperature and the raw materials are put in, the time starts to be calculated. When the feeding action of the next process occurs, if the calculated time does not reach the specified time of the step, an alarm will be sounded; when the calculated time exceeds the specified time of the step, an alarm will be sounded regardless of whether the raw materials are put into the wok.

[0035] This embodiment monitors the medium temperature, the staff's material taking and material placing behaviors, the types of raw materials taken away, and the frying time, and makes early warning judgments on the behavior of not putting raw materials in accordance with the requirements during the frying process, thereby achieving real-time supervision of the food frying process, timely reminding the operator to correct the operation behavior, reducing the occurrence of improper timing or wrong order of putting raw materials, and reducing the waste of raw materials. It improves production efficiency and the stability of product quality.

[0036] In this embodiment, the early warning judgment results, image data, behavioral characteristics, alarm signals and other data can also be uploaded to the remote monitoring platform, thereby realizing remote supervision of the on-site operation process, so as to make targeted decisions on erroneous operation behaviors in a timely manner.

[0037] Embodiment 2: This embodiment provides a process monitoring system to execute the process monitoring method described in Embodiment 1. Figure 2 As shown, the process monitoring system includes a process monitoring module 600, and the process monitoring module includes: The data acquisition unit 610 is used to acquire the ultrasonic transmitting node, the ultrasonic receiving node, the first image information, the second image information and the weight data information; The temperature calculation unit 620 is used to calculate the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node to obtain the medium temperature. The specific calculation method can refer to step S110 to step S130 recorded in embodiment 1; The first behavior analysis unit 630 is used to process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain the first behavior feature. The specific analysis method can refer to steps S210 to S220 described in Example 1; A first early warning judgment unit 670 is used to perform a first early warning judgment according to the medium temperature and the first behavior characteristic, obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result; The second behavior analysis unit 650 is used to obtain second image information, process the second image information to obtain a second motion feature, and perform a second behavior feature analysis based on the second motion feature to obtain a second behavior feature. The specific analysis method can refer to steps S410 to S420 described in Example 1; The raw material type judgment unit 660 is used to obtain the weight data information of the preset position, perform weight change judgment according to the weight data information, obtain the weight change judgment result, perform the removal of raw material type matching according to the weight change judgment result, and obtain the removal of raw material type matching result. A second warning judgment unit 690 is used to perform a second warning judgment according to the second behavior feature and the matching result of the taken-away raw material category, obtain a second warning judgment result, and generate a second alarm signal based on the second warning judgment result; A timing unit 640 performs a timing judgment according to the first warning judgment result, performs timing based on the timing judgment result, and obtains a timing result; The third warning judgment unit 680 is used to perform a third warning judgment according to the timing result and the first behavior feature, obtain a third warning judgment result, and generate a third alarm signal based on the third warning judgment result.

[0038] In this embodiment, the process monitoring system may further include: Ultrasonic transmitting module 100, used for transmitting ultrasonic waves; The ultrasonic receiving module 200 is used to receive the ultrasonic waves emitted by the ultrasonic transmitting module; An image acquisition module 300, which is disposed in the first preset area and the second preset area, and is used to acquire a first image in the first preset area and a second image in the second preset area; The weight sensor module 400 is arranged at a preset position and is used to collect weight data information at the preset position.

[0039] In addition, the process monitoring system may also include a remote monitoring platform 500, which can receive the early warning judgment results, image data, behavioral characteristics, alarm signals and other data sent by the process monitoring module, and output the above data to a designated window for display. At the same time, the remote monitoring platform can also input product processing flow information to the process monitoring module through the remote monitoring platform, so that the process monitoring module can decompose the processing flow information and extract keywords to form unit processing steps, so that the process monitoring module can supervise the on-site process in accordance with standardized operation requirements based on the unit processing steps.

[0040] The process monitoring system may further include an alarm module 700, which may receive alarm signals generated by the first, second, and third early warning judgment modules, and thereby execute at least one alarm action such as sound, light, text message, modular warning, etc. according to the alarm signal.

[0041] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A process monitoring method, characterized in that: include: Obtaining an ultrasonic transmitting node and an ultrasonic receiving node, and calculating a medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node; Acquire first image information, process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain a first behavior feature; Perform a first early warning judgment according to the medium temperature and the first behavior characteristic, obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result; Acquire second image information, process the second image information to obtain a second motion feature, and perform a second behavior feature analysis based on the second motion feature to obtain a second behavior feature; Obtaining weight data information of a preset position, performing weight change judgment according to the weight data information, obtaining a weight change judgment result, performing a category match of the taken-away raw materials according to the weight change judgment result, and obtaining a category match result of the taken-away raw materials; Perform a second early warning judgment based on the second behavior feature and the matching result of the taken-away raw material category, obtain a second early warning judgment result, and generate a second alarm signal based on the second early warning judgment result; Performing a timing judgment according to the first warning judgment result, and performing timing based on the timing judgment result to obtain a timing result; A third early warning judgment is performed according to the timing result and the first behavior feature to obtain a third early warning judgment result, and a third alarm signal is generated based on the third early warning judgment result.

2. A process monitoring method according to claim 1, characterized in that: The method of calculating the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node includes: Calculate the ultrasonic flight time according to the ultrasonic transmitting node and the ultrasonic receiving node; The single-point medium temperature is calculated based on the ultrasonic flight time to obtain the single-point medium temperature; The medium temperature is calculated based on the single-point medium temperature to obtain the medium temperature.

3. A process monitoring method according to claim 1, characterized in that: The performing a first behavior feature analysis based on the first motion feature to obtain the first behavior feature includes: Extracting a first target feature from the first motion feature, and performing first target feature motion calculation based on the first target feature; A first behavior match is performed according to a result of the first target feature motion calculation to obtain a first behavior feature.

4. A process monitoring method according to claim 3, characterized in that: The performing first target feature motion calculation based on the first target feature includes: Based on the first target feature, extracting a first target feature point in the first target feature; Calculate the vector of the first target feature point between adjacent frames to obtain a unit first target feature vector; Calculate the first target rotation angle according to the unit first target characteristic vector to obtain the first target rotation angle; The performing first behavior matching according to the result of the first target feature motion calculation to obtain the first behavior feature includes: According to the unit first target feature vector and the first target feature rotation angle, a match is performed with the vector, rotation angle-behavior corresponding data sample set. If a match is made, the first behavior is established, otherwise it is not established.

5. A process monitoring method according to claim 1, characterized in that: The performing a second behavior feature analysis based on the second motion feature to obtain the second behavior feature includes: Extracting a second target feature from the second motion feature, and performing second target feature motion calculation based on the second target feature; A second behavior match is performed based on the result of the second target feature motion calculation to obtain a second behavior feature.

6. A process monitoring method according to claim 5, characterized in that: The performing motion calculation of the second target feature based on the second target feature includes: Based on the second target feature, extracting a second target feature point in the second target feature; Calculate the vector of the second target feature point between adjacent frames to obtain a unit second target feature vector; Calculate the displacement of the second target feature point between adjacent frames to obtain a unit second target displacement; Calculate the second target displacement amount according to the unit second target displacement amount to obtain the second target displacement amount; The performing second behavior matching according to the result of the second target feature motion calculation includes: According to the unit second target characteristic vector and the second target displacement, the vector, displacement-behavior corresponding data sample set is matched. If they match, the second behavior is established, otherwise it is not established.

7. A process monitoring system, characterized in that: The process monitoring system is used to execute a process monitoring method according to any one of claims 1 to 6, and the process monitoring system comprises: A data acquisition unit, used to obtain an ultrasonic transmitting node, an ultrasonic receiving node, first image information, second image information, and weight data information; A temperature calculation unit, used for calculating the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node; A first behavior analysis unit is used to process the first image information to obtain a first motion feature, and perform a first behavior feature analysis based on the first motion feature to obtain the first behavior feature; A first early warning judgment unit, used to perform a first early warning judgment according to the medium temperature and the first behavior characteristic, obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result; A second behavior analysis unit is used to obtain second image information, process the second image information to obtain a second motion feature, and perform a second behavior feature analysis based on the second motion feature to obtain a second behavior feature; A raw material type judgment unit is used to obtain weight data information of a preset position, perform weight change judgment according to the weight data information, obtain a weight change judgment result, perform a category match of the raw materials to be removed according to the weight change judgment result, and obtain a category match result of the raw materials to be removed; A second early warning judgment unit is used to perform a second early warning judgment according to the second behavior feature and the matching result of the category of the taken raw materials, obtain a second early warning judgment result, and generate a second alarm signal based on the second early warning judgment result; A timing unit, used to perform a timing judgment according to the first warning judgment result, perform timing based on the timing judgment result, and obtain a timing result; The third early warning judgment unit is used to perform a third early warning judgment according to the timing result and the first behavior feature, obtain a third early warning judgment result, and generate a third alarm signal based on the third early warning judgment result.

Images