A process monitoring method and system thereof

Abstract

The present invention discloses a process monitoring method and system thereof, relating to the field of automatic control. The process monitoring method includes calculating the medium temperature according to an ultrasonic emission node and an ultrasonic reception node, and processing first image information to obtain a first behavior feature; making a first early warning judgment according to the medium temperature and the first behavior feature; processing second image information to obtain a second behavior feature; acquiring weight data information, making a weight change judgment according to the weight data information, and matching the removed raw material category according to the weight change judgment result; making a second early warning judgment according to the second behavior feature and the matching result of the removed raw material category; making a timing judgment according to the first early warning judgment result, timing based on the timing judgment result, and making a third early warning judgment according to the timing result and the first behavior feature. Through the present invention, it is possible to make an early warning judgment on the behavior of not putting raw materials according to requirements during the frying process, so as to realize real-time supervision of the food frying process.

Description

Technical Field

[0001] The present invention relates to the field of automatic control, and particularly to a process monitoring method and system thereof. Background Art

[0002] In the food frying process, such as the frying process of hot pot base, the control of oil temperature, the order of raw material feeding, and the grasp of frying time are all factors affecting product quality. The existing frying equipment on the market, although to a certain extent realizes the automation of the frying process and can display the frying process to the processing personnel through a visual window so that they can operate according to the established process, when performing the frying task, it is inevitable that the timing of raw material feeding is improper or the feeding order is wrong. These subtle deviations may cause a chain reaction during the frying process, resulting in the final fried product not meeting the expected processing requirements, thereby causing waste of raw materials. This not only increases production costs but also affects production efficiency and the stability of product quality. Summary of the Invention

[0003] Therefore, to solve the above deficiencies, the present invention is hereby proposed. Through the present invention, it is possible to perform early warning judgment on the behavior of not feeding raw materials as required during the frying process, thereby realizing real-time supervision of the food frying process, timely reminding the operator to correct the operation behavior, reducing the occurrence of improper raw material feeding timing or wrong feeding order, and reducing waste of raw materials.

[0004] On the one hand, the present invention hereby proposes a process monitoring method, including:

[0005] Obtain an ultrasonic emission node and an ultrasonic reception node, and calculate the medium temperature according to the ultrasonic emission node and the ultrasonic reception node;

[0006] Obtain first image information, process the first image information to obtain a first motion feature, perform first behavior feature analysis based on the first motion feature, and obtain a first behavior feature;

[0007] Perform a first early warning judgment according to the medium temperature and the first behavior feature to obtain a first early warning judgment result, and generate a first alarm signal according to the first early warning judgment result;

[0008] Obtain second image information, process the second image information to obtain a second motion feature, perform second behavior feature analysis based on the second motion feature, and obtain a second behavior feature; specifically, the second image information is dynamic image information of the raw material placement point;

[0009] Obtain the weight data information at a preset position, perform a weight change judgment according to the weight data information to obtain a weight change judgment result, and perform matching of the removed raw material category according to the weight change judgment result to obtain a matching result of the removed raw material category;

[0010] Perform a second warning judgment based on the matching result of the characteristics in the second row and the category of the removed raw materials to obtain a second warning judgment result, and generate a second alarm signal based on the second warning judgment result;

[0011] Perform a timing judgment based on the first warning judgment result, perform timing based on the timing judgment result, and obtain a timing result;

[0012] Perform a third warning judgment based on the timing result and the first behavior characteristics to obtain a third warning judgment result, and generate a third alarm signal based on the third warning judgment result.

[0013] Further, the calculating the medium temperature according to the ultrasonic transmitting node and the ultrasonic receiving node includes:

[0014] Calculate the ultrasonic flight time according to the ultrasonic transmitting node and the ultrasonic receiving node;

[0015] Perform a single-point medium temperature calculation based on the ultrasonic flight time to obtain a single-point medium temperature;

[0016] Perform a medium temperature calculation according to the single-point medium temperature to obtain the medium temperature.

[0017] Further, the analyzing the first behavior characteristics based on the motion characteristics to obtain the first behavior characteristics includes:

[0018] Extract the first target feature in the first motion feature, and perform a first target feature motion calculation based on the first target feature;

[0019] Perform a first behavior matching according to the result of the first target feature motion calculation to obtain the first behavior characteristics.

[0020] Further, the performing a first target feature motion calculation based on the first target feature includes:

[0021] Based on the first target feature, extract the first target feature points in the first target feature;

[0022] Calculate the vector of the first target feature points between adjacent frames to obtain a unit first target feature vector;

[0023] Calculate the first target rotation angle according to the unit first target feature vector to obtain the first target rotation angle;

[0024] The performing a first behavior matching according to the result of the first target feature motion calculation to obtain the first behavior characteristics includes:

[0025] Match the first target feature vector and the first target feature rotation angle with the vector and rotation angle - behavior corresponding data sample set. If there is a match, the first behavior is established; otherwise, it is not established.

[0026] Further, the second behavior feature analysis based on the second motion feature to obtain the second behavior feature includes:

[0027] Extract the second target feature in the second motion feature, and based on the second target feature, perform second target feature motion calculation;

[0028] Perform second behavior matching according to the result of the second target feature motion calculation to obtain the second behavior feature.

[0029] Further, the second target feature motion calculation based on the second target feature includes:

[0030] Based on the second target feature, extract the second target feature points in the second target feature;

[0031] Calculate the vector of the second target feature points between adjacent frames to obtain the unit second target feature vector;

[0032] Calculate the displacement of the second target feature points between adjacent frames to obtain the unit second target displacement;

[0033] Calculate the second target displacement according to the unit second target displacement to obtain the second target displacement;

[0034] The second behavior matching according to the result of the second target feature motion calculation includes:

[0035] Match the unit second target feature vector and the second target displacement with the vector and displacement - behavior corresponding data sample set. If there is a match, the second behavior is established; otherwise, it is not established.

[0036] On the other hand, the present invention also provides a process monitoring system, which is used to execute the process monitoring method described above, and the process monitoring system includes:

[0037] A data acquisition unit for acquiring ultrasonic emission nodes, ultrasonic reception nodes, first image information, second image information, and weight data information;

[0038] A temperature calculation unit for calculating the medium temperature according to the ultrasonic emission node and the ultrasonic reception node;

[0039] A first behavior analysis unit for processing the first image information to obtain the first motion feature, and performing first behavior feature analysis based on the first motion feature to obtain the first behavior feature;

[0040] The first early warning judgment unit is used to perform the first early warning judgment based on the medium temperature and the first behavior characteristics, obtain the first early warning judgment result, and generate the first alarm signal according to the first early warning judgment result;

[0041] The second behavior analysis unit is used to obtain the second image information, process the second image information to obtain the second motion characteristics, and perform the second behavior characteristic analysis based on the second motion characteristics to obtain the second behavior characteristics;

[0042] The raw material type judgment unit is used to obtain the weight data information at a preset position, perform a weight change judgment according to the weight data information to obtain the weight change judgment result, and perform a matching of the removed raw material category according to the weight change judgment result to obtain the matching result of the removed raw material category;

[0043] The second early warning judgment unit is used to perform the second early warning judgment based on the second behavior characteristics and the matching result of the removed raw material category, obtain the second early warning judgment result, and generate the second alarm signal based on the second early warning judgment result;

[0044] The timing unit performs a timing judgment according to the first early warning judgment result and performs timing based on the timing judgment result to obtain the timing result;

[0045] The third early warning judgment unit is used to perform the third early warning judgment based on the timing result and the first behavior characteristics, obtain the third early warning judgment result, and generate the third alarm signal based on the third early warning judgment result.

[0046] The present invention has the following advantages:

[0047] By monitoring the medium temperature, the behaviors of the staff in taking and placing raw materials, the type of the removed raw materials, and the frying time, the present invention performs an early warning judgment on the behavior of not putting raw materials according to requirements during the frying process, so as to realize real-time supervision of the food frying process, timely remind the operator to correct the operation behavior, reduce the occurrence of improper raw material feeding time or wrong feeding order, reduce the waste of raw materials, and improve the production efficiency and the stability of product quality. Description of the Drawings

[0048] Figure 1 is a schematic flowchart of the process monitoring method;

[0049] Figure 2 is a schematic logical structure diagram of the process monitoring system;

[0050] Figure 3 is Figure 2 a schematic logical structure diagram of the process monitoring module in the process monitoring system shown;

[0051] 100. Ultrasonic emission module;

[0052] 200. Ultrasonic receiving module;

[0053] 300. Image acquisition module;

[0054] 400. Weight sensing module;

[0055] 500. Remote monitoring platform;

[0056] 600. Process monitoring module; 610. Data acquisition unit; 620. Temperature calculation unit; 630. First line analysis unit; 640. Timing unit; 650. Second line analysis unit; 660. Raw material type judgment unit; 670. First warning judgment unit; 680. Third warning judgment unit; 690. Second warning judgment unit;

[0057] 700. Alarm module. Detailed implementation manners

[0058] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application.

[0059] In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0060] As described in the background art, for the frying equipment existing in the current market, although it realizes the automation of the frying process to a certain extent and can display the frying process to the processing personnel through a visual window so that they can operate according to the established process, when performing the frying task, it is inevitable that the raw materials are put in at an inappropriate time or in the wrong order. These slight deviations may cause a chain reaction during the frying process, resulting in the final fried product not meeting the expected processing requirements, thereby causing waste of raw materials. This not only increases the production cost, but also affects the production efficiency and the stability of product quality.

[0061] Embodiment 1:

[0062] Therefore, in order to solve the above technical problems existing in the prior art, a process monitoring method is proposed here. The monitoring method includes:

[0063] S100: Obtain an ultrasonic emission node and an ultrasonic reception node, and calculate the medium temperature according to the ultrasonic emission node and the ultrasonic reception node;

[0064] Specifically, the calculating the medium temperature according to the ultrasonic emission node and the ultrasonic reception node includes:

[0065] S110: Calculate the ultrasonic flight time according to the ultrasonic emission node and the ultrasonic reception node. In this embodiment, the ultrasonic generator and the ultrasonic receiver are arranged corresponding to each other, and at least one set of the ultrasonic generator and the ultrasonic receiver is provided. The specific calculation method of the ultrasonic flight time is as follows:

[0066] ;

[0067] Wherein, is the ultrasonic flight time, in seconds; is the ultrasonic emission node, in seconds; is the ultrasonic reception node, in seconds; n is the number of the ultrasonic transmitter and the corresponding ultrasonic receiver.

[0068] S120: Perform single-point medium temperature calculation based on the ultrasonic flight time to obtain the single-point medium temperature;

[0069] Specifically, the single-point medium temperature calculation method is as follows:

[0070] ;

[0071] Wherein, is the single-point medium temperature at the position point where the th group of ultrasonic transmitters and ultrasonic receivers are located, in degrees Celsius; is the distance between the th group of ultrasonic transmitters and ultrasonic receivers, in meters; is the propagation speed of sound waves in the medium at 0 degrees Celsius, in meters per second; is the change amount of the sound wave speed when the medium temperature changes, in meters per second per degree Celsius; , are both determined by the properties of the medium;

[0072] S130: Calculate the medium temperature according to the single-point medium temperature to obtain the medium temperature;

[0073] Specifically, the medium temperature calculation method is as follows:

[0074] ;

[0075] Wherein, is the medium temperature, with the unit of degree Celsius.

[0076] S200: Obtain the first image information, process the first image information to obtain the first motion feature, and perform the first behavior feature analysis based on the first motion feature to obtain the first behavior feature;

[0077] Specifically, the first image information can be obtained by an image capture device shooting the image at a first preset position. For example, a camera, a video camera, etc. In this embodiment, the first preset position is the position where the wok is located, and the first image information is the dynamic image information of the position where the wok is located. In this embodiment, the first image information is processed through an image recognition algorithm (such as a moving object detection algorithm based on RLBSP texture and HSV), including background model initialization, foreground detection, pixel classification, separation of foreground and background, etc., so as to separate the motion features in the first image. In this embodiment, the first motion feature is a container filled with raw materials.

[0078] In this embodiment, the performing the first behavior feature analysis based on the first motion feature to obtain the first behavior feature includes:

[0079] S210: Extract the first target feature in the first motion feature, and perform the first target feature motion calculation based on the first target feature;

[0080] Specifically, the specific calculation method of the first target feature motion calculation is as follows:

[0081] S211: Based on the first target feature, extract the first target feature points in the first target feature;

[0082] S212: Calculate the vector of the first target feature points between adjacent frames to obtain the unit first target feature vector. The calculation method is specifically as follows:

[0083] ;

[0084] where m is the number of frames, is the unit first target feature vector; , is the horizontal and vertical coordinates of the first target feature point in the -th frame of the first image, , is the horizontal and vertical coordinates of the first target feature point in the m -th frame of the first image.

[0085] S213: Calculate the first target rotation angle according to the unit first target feature vector to obtain the first target rotation angle. The calculation method is specifically as follows:

[0086] ;

[0087] wherein, is the first target rotation angle, C is the number of first target feature points.

[0088] S220: Perform the first row matching based on the result calculated from the first target feature motion to obtain the first row feature. Specifically, match according to the unit first target feature vector and the first target feature rotation angle with the vector, rotation angle - behavior corresponding data sample set. If the match is successful, the first row is established; otherwise, it is not.

[0089] S300: Perform the first warning judgment based on the medium temperature and the first row feature to obtain the first warning judgment result, and generate the first alarm signal according to the first warning judgment result;

[0090] Specifically, when the first row is established, if the medium temperature is lower than or higher than the preset temperature range at this time, it is judged as "not operating according to the requirements", and an alarm signal is generated at this time; when the first row is not established, if the medium temperature is higher than the preset temperature range at this time, it is also judged as "not operating according to the requirements", and the first alarm signal is generated; the first alarm signal is output to the alarm module so that the alarm module alarms in at least one of the ways including but not limited to sound, light, text message, etc. When the first row is established and the medium temperature is within the preset temperature range, it is judged as "operating according to the requirements", and the first alarm signal is not generated at this time; through the above method, the occurrence probability of putting raw materials into the frying equipment when the oil temperature has not reached the preset temperature or is higher than the preset temperature can be reduced.

[0091] Exemplarily, taking the frying of hot pot base as an example, before frying, cooking oil needs to be injected into the frying pan, and after heating the cooking oil to the preset temperature, the raw materials will be put in sequence. During this process, at least one set of ultrasonic emission module and ultrasonic receiving module arranged on the side of the frying pan are used to measure the temperature of the cooking oil. The specific measurement method can refer to step S110 to step S130 to obtain the current temperature of the cooking oil. At the same time, the image of the position area of the frying pan is collected by the image acquisition module (such as a camera). When the first motion feature (i.e., the container containing raw materials) is recognized in the image, the rotation angle of the container is monitored and calculated in real time through steps S211 to S213. When the rotation angle of the container reaches a certain angle, it is determined that the first behavior (i.e., the feeding behavior) is established. If the temperature has not reached the preset temperature or is higher than the preset temperature at this time, the first alarm signal is generated. After receiving the first alarm signal, the alarm module starts to alarm.

[0092] S400: Obtain the second image information, process the second image information to obtain the second motion feature, and perform second behavior feature analysis based on the second motion feature to obtain the second behavior feature; specifically, the second image information is the dynamic image information of the raw material placement point. In this embodiment, the second motion feature is the container containing the raw material.

[0093] In this embodiment, the performing second behavior feature analysis based on the motion feature to obtain the second behavior feature includes:

[0094] S410: Extract the second target feature in the second motion feature, and perform second target feature motion calculation based on the second target feature;

[0095] Specifically, the specific calculation method of the second target feature motion calculation is as follows:

[0096] S411: Based on the second target feature, extract the second target feature points in the second target feature;

[0097] S412: Calculate the vector of the second target feature points between adjacent frames to obtain the unit second target feature vector. The specific calculation method can refer to the calculation method of the unit first target feature vector;

[0098] S413: Calculate the displacement of the second target feature points between adjacent frames to obtain the unit second target displacement amount. The specific calculation method is as follows:

[0099] ;

[0100] where is the unit second target displacement amount; is the abscissa of the second target feature point in the second image of the m +1th frame; is the abscissa of the second target feature point in the second image of the m th frame;

[0101] S414: Calculate the second target displacement amount. The specific calculation method is as follows:

[0102] ; [[ID=4�]]

[0103] where W is the second target displacement amount.

[0104] S420: Perform second behavior matching according to the result of the second target feature motion calculation to obtain the second behavior feature. Specifically, the unit second target feature vector and the second target displacement amount are matched with the vector, displacement amount - behavior corresponding data sample set. If the match is successful, the second behavior is established; otherwise, it is not established.

[0105] S500: Obtain the weight data information of a preset position, make a weight change judgment based on the weight data information to obtain a weight change judgment result, and perform a matching of the removed raw material category according to the weight change judgment result to obtain a matching result of the removed raw material category;

[0106] Specifically, the preset position can be a raw material placement platform. Containers containing a certain weight of raw materials can be placed on this raw material placement platform. A weight sensing module can be set at the preset position. This weight sensing module can be bound to the raw material placement platform to monitor the weight data information of the placement platform in real time. Since each placement platform is used to place specific raw materials, when the container at the preset position is removed, the weight of the placement platform changes. At this time, the raw material category can be matched according to the number, mark, etc. of the placement platform where the weight change occurs. The number or mark-category data table can be matched with the number or mark of the placement platform where the weight change occurs to match the removed raw material category.

[0107] S600: Perform a second warning judgment based on the second behavior feature and the matching result of the removed raw material category to obtain a second warning judgment result, and generate a second alarm signal based on the second warning judgment result.

[0108] Specifically, when the second behavior is established, if the removed raw material category does not match the raw material in the current step, a second alarm signal is generated; if the removed raw material category matches the raw material in the current step, no second alarm signal is generated; in the above manner, when the wrong raw material is taken, a reminder can be given in time, thereby reducing the probability of misplacing raw materials.

[0109] Exemplarily, taking the frying of hot pot base as an example, the raw materials for frying are usually placed in specific areas or platforms. Generally, the raw materials are pre-poured into containers, weighed, and then transported to these areas or platforms. Different raw materials are placed on the corresponding platforms. The images of each area or platform are collected by an image acquisition module. When taking raw materials, the movement direction and movement displacement of the container are monitored and calculated through steps S411 to S414. When the movement direction conforms to the preset movement direction and the movement displacement reaches the preset displacement amount, it is determined that the second behavior is established (i.e., the material taking behavior is established). At the same time, by calculating the weight before and after of each platform or area, it is judged whether the weight of each platform or area has changed, and the type of raw material is matched according to the number or mark of each platform or area where the weight change occurs. For example, the number of the platform where the weight change occurs is 2, and the type of raw material corresponding to platform number 2 in the number-category data table is ginger. At this time, it is judged that the removed raw material is ginger, and the removed raw material is matched with the raw material recorded in the current step. If they do not match, an alarm is given.

[0110] S700: Perform a timing judgment based on the first warning judgment result, and perform timing based on the timing judgment result to obtain a timing result;

[0111] Specifically, when the first warning judgment result is "the operation is not performed as required", the timing unit determines that the timing operation cannot be performed. If the first warning judgment result is "the operation is performed as required", the timing unit determines that the timing operation can be performed and performs the timing operation.

[0112] S800: Perform a third warning judgment based on the timing result (the timing result can be the length of time) and the first behavior feature to obtain a third warning judgment result, and generate a third alarm signal based on the third warning judgment result.

[0113] Specifically, when the medium temperature reaches the preset temperature, at this time, the raw material is put into the medium and timing starts. When the first behavior of the next process occurs and the timing does not reach the preset time, it is judged as "not meeting the requirements" and an alarm signal is generated; after the timing exceeds the preset time, regardless of whether the first behavior occurs or not, it is judged as "not meeting the requirements" and an alarm signal is generated; when the first behavior of the next process occurs and the timing reaches the preset time, it is judged as "meeting the requirements" and no alarm signal is generated. By the above method, the situation of insufficient or excessive frying time of the raw material can be reduced.

[0114] Exemplarily, when the temperature reaches the preset temperature and the raw material is put in, the time starts to be calculated at this time. After the feeding action of the next process occurs, if the calculated time does not reach the time specified in the step, an alarm is given; when the calculated time exceeds the time specified in the step, regardless of whether the raw material is put into the frying pan or not, an alarm will be given.

[0115] In this embodiment, by monitoring the medium temperature, the actions of the staff in taking and placing materials, the type of raw materials taken away, and the frying time, a warning judgment is made on the behavior of not putting raw materials as required during the frying process, so as to realize real-time supervision of the food frying process, timely remind the operator to correct the operation behavior, reduce the situation of improper raw material feeding time or wrong feeding order, and reduce the waste of raw materials. Improve the production efficiency and the stability of product quality.

[0116] In this embodiment, data such as the warning judgment result, image data, behavior feature, alarm signal, etc. can also be uploaded to the remote monitoring platform, so as to realize remote supervision of the on-site operation process, so as to make targeted decisions on misoperation behaviors in a timely manner.

[0117] Embodiment 2:

[0118] This embodiment provides a process monitoring system here to execute the process monitoring method described in Embodiment 1, as Figure 2 shown. The process monitoring system includes a process monitoring module 600, and the process monitoring module includes:

[0119] The data acquisition unit 610 is used to obtain the ultrasonic emission node, the ultrasonic reception node, the first image information, the second image information, and the weight data information;

[0120] The temperature calculation unit 620 is used to calculate the medium temperature based on the ultrasonic emission node and the ultrasonic reception node, obtain the medium temperature, and the specific calculation method can refer to steps S110 to S130 recorded in Embodiment 1;

[0121] The first behavior analysis unit 630 is used to process the first image information to obtain the first motion feature, perform the first behavior feature analysis based on the first motion feature, and obtain the first behavior feature. The specific analysis method can refer to steps S210 to S220 recorded in Embodiment 1;

[0122] The first warning judgment unit 670 is used to perform the first warning judgment according to the medium temperature and the first behavior feature, obtain the first warning judgment result, and generate the first alarm signal according to the first warning judgment result;

[0123] The second behavior analysis unit 650 is used to obtain the second image information, process the second image information to obtain the second motion feature, perform the second behavior feature analysis based on the second motion feature, and obtain the second behavior feature. The specific analysis method can refer to steps S410 to S420 recorded in Embodiment 1;

[0124] The raw material type judgment unit 660 is used to obtain the weight data information at the preset position, perform the weight change judgment according to the weight data information, obtain the weight change judgment result, and perform the matching of the removed raw material category according to the weight change judgment result to obtain the matching result of the removed raw material category

[0125] The second warning judgment unit 690 is used to perform the second warning judgment according to the second behavior feature and the matching result of the removed raw material category, obtain the second warning judgment result, and generate the second alarm signal based on the second warning judgment result;

[0126] The timing unit 640 performs the timing judgment according to the first warning judgment result, performs the timing based on the timing judgment result, and obtains the timing result;

[0127] The third warning judgment unit 680 is used to perform the third warning judgment according to the timing result and the first behavior feature, obtain the third warning judgment result, and generate the third alarm signal based on the third warning judgment result.

[0128] In this embodiment, the process monitoring system may further include:

[0129] The ultrasonic emission module 100 is used to emit ultrasonic waves;

[0130] The ultrasonic receiving module 200 is used to receive the ultrasonic waves emitted by the ultrasonic transmitting module;

[0131] The image acquisition module 300 is arranged in the first preset area and the second preset area. The first image of the first preset area and the second image of the second preset area are acquired through this image acquisition module;

[0132] The weight sensing module 400 is arranged at a preset position and is used to collect the weight data information at the preset position.

[0133] In addition, the process monitoring system may further include a remote monitoring platform 500. This remote monitoring platform can receive data such as the early warning judgment result, image data, behavior characteristics, alarm signal, etc. sent by the process monitoring module, and output the above data to a specified window for display. At the same time, the product processing flow information can also be input into the process monitoring module through the remote monitoring platform, so that the process monitoring module decomposes the processing flow information and extracts keywords to form unit processing steps, so that the process monitoring module can supervise the on-site process based on the unit processing steps according to the standardized operation requirements.

[0134] The process monitoring system may further include an alarm module 700. This alarm module can receive the alarm signals generated by the first, second, and third early warning judgment modules, and thus perform at least one alarm behavior such as sound, light, short message, modular warning, etc. according to the alarm signals.

[0135] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A process monitoring method, characterized in that, Including: Obtain an ultrasonic emission node and an ultrasonic reception node, and calculate the medium temperature according to the ultrasonic emission node and the ultrasonic reception node; Obtain first image information, process the first image information to obtain a first motion feature, perform a first behavior feature analysis based on the first motion feature, and obtain a first behavior feature; Perform a first warning judgment according to the medium temperature and the first behavior feature, obtain a first warning judgment result, and generate a first alarm signal according to the first warning judgment result; Obtain second image information, process the second image information to obtain a second motion feature, perform a second behavior feature analysis based on the second motion feature, and obtain a second behavior feature; Obtain the weight data information at a preset position, perform a weight change judgment according to the weight data information, obtain a weight change judgment result, and perform a matching of the removed raw material category according to the weight change judgment result to obtain a matching result of the removed raw material category; Perform a second warning judgment according to the second behavior feature and the matching result of the removed raw material category, obtain a second warning judgment result, and generate a second alarm signal based on the second warning judgment result; Perform a timing judgment according to the first warning judgment result, perform timing based on the timing judgment result, and obtain a timing result; Perform a third warning judgment according to the timing result and the first behavior feature, obtain a third warning judgment result, and generate a third alarm signal based on the third warning judgment result; The performing a first behavior feature analysis based on the first motion feature to obtain a first behavior feature includes: Based on a first target feature, extract first target feature points in the first target feature; Calculate the vectors of the first target feature points between adjacent frames to obtain a unit first target feature vector; Calculate a first target rotation angle according to the unit first target feature vector to obtain a first target rotation angle; Match the unit first target feature vector and the first target feature rotation angle with a vector and rotation angle-behavior corresponding data sample set. If they match, the first behavior is established; otherwise, it is not established; The performing a second warning judgment according to the second behavior feature and the matching result of the removed raw material category, obtaining a second warning judgment result, and generating a second alarm signal based on the second warning judgment result includes: When the first behavior is established, if the medium temperature is lower than or higher than the preset temperature range at this time, it is judged that the operation is not performed according to the requirements; an alarm signal is generated at this time; When the first behavior is not established, if the medium temperature is higher than the preset temperature range at this time, it is judged that the operation is not performed according to the requirements; a first alarm signal is generated; When the first behavior is established and the medium temperature is within the preset temperature range, it is judged that the operation is performed according to the requirements; a first alarm signal is not generated at this time; The calculation method of the medium temperature is as follows: ； Wherein, is the medium temperature in degrees Celsius; is the single-point medium temperature at the position points where the th group of ultrasonic transmitters and ultrasonic receivers are located, in degrees Celsius; n is the number of ultrasonic transmitters and the corresponding ultrasonic receivers; The calculation method of the single-point medium temperature is as follows: ； Among them, is the distance between the group of ultrasonic transmitters and ultrasonic receivers, in meters; is the speed of sound in the medium at 0 degrees Celsius, in meters per second; is the change in the speed of sound in the medium with temperature change, in meters per second per degree Celsius; is the ultrasonic flight time, in seconds; the ultrasonic flight time is determined according to the ultrasonic emission node and the ultrasonic reception node; The calculation method of the unit first target feature vector is as follows: ； Among them, m is the number of frames, is the unit first target feature vector; , is the horizontal and vertical coordinates of the first target feature point in the th frame of the first image, , is the horizontal and vertical coordinates of the first target feature point in the m th frame of the first image; The specific calculation method of the first target rotation angle is as follows: ； Among them, is the first target rotation angle, C is the number of first target feature points.

2. The process monitoring method according to claim 1, wherein The performing a second behavior feature analysis based on the second motion feature to obtain a second behavior feature includes: Extract a second target feature in the second motion feature, and perform second target feature motion calculation based on the second target feature; Perform second behavior matching based on the result of the second target feature motion calculation to obtain second behavior features.

3. The process monitoring method according to claim 2, characterized in that Performing the second target feature motion calculation based on the second target feature includes: Based on the second target feature, extract the second target feature points in the second target feature; Calculate the vector of the second target feature points between adjacent frames to obtain the unit second target feature vector; Calculate the displacement of the second target feature points between adjacent frames to obtain the unit second target displacement; Calculate the second target displacement based on the unit second target displacement to obtain the second target displacement; The performing second behavior matching according to the result of the second target feature motion calculation includes: Match the unit second target feature vector and the second target displacement with the vector, displacement-behavior corresponding data sample set. If the match is successful, the second behavior is established; otherwise, it is not.

4. A process monitoring system, characterized in that, This process monitoring system is used to execute a process monitoring method as described in any one of claims 1 to 3. The process monitoring system includes: A data acquisition unit for acquiring ultrasonic emission nodes, ultrasonic reception nodes, first image information, second image information, and weight data information; A temperature calculation unit for calculating the medium temperature based on the ultrasonic emission node and the ultrasonic reception node; A first behavior analysis unit for processing the first image information to obtain a first motion feature, and performing first behavior feature analysis based on the first motion feature to obtain a first behavior feature; A first warning judgment unit for performing a first warning judgment based on the medium temperature and the first behavior feature to obtain a first warning judgment result, and generating a first alarm signal according to the first warning judgment result; A second behavior analysis unit for acquiring the second image information, processing the second image information to obtain a second motion feature, and performing second behavior feature analysis based on the second motion feature to obtain a second behavior feature; A raw material type judgment unit for acquiring the weight data information at a preset position, performing a weight change judgment based on the weight data information to obtain a weight change judgment result, and performing a removed raw material category matching based on the weight change judgment result to obtain a removed raw material category matching result; A second warning judgment unit for performing a second warning judgment based on the second behavior feature and the removed raw material category matching result to obtain a second warning judgment result, and generating a second alarm signal based on the second warning judgment result; A timing unit for performing a timing judgment based on the first warning judgment result, and performing timing based on the timing judgment result to obtain a timing result; A third warning judgment unit for performing a third warning judgment based on the timing result and the first behavior feature to obtain a third warning judgment result, and generating a third alarm signal based on the third warning judgment result.

Images