Automated Material Deviation Correction System and Method Based on Computer Vision and Feedback Control

Abstract

The present invention discloses a material automatic deviation correction system and method based on computer vision and feedback control, including: a material state acquisition module for acquiring production state data of the material; a material state data preprocessing module for preprocessing the acquired production state data of the material to filter out unnecessary information to reduce the calculation amount; a material state information calculation module for calculating the edge position information of the material and calculating the width information of the material; a multi-level negative feedback control module for calculating a deviation correction instruction for the current moment of material production; and a central service module for receiving, storing, and managing the production state data of the material and the corresponding control commands. The present invention can effectively detect the edge position condition and the width of the material in the material production environment and achieve a detection effect at the millimeter level.

Description

Technical Field

[0001] The present invention belongs to the fields of industrial Internet and computer vision, and particularly relates to a material automatic deviation correction system and method based on computer vision and feedback control. Background Art

[0002] As a product of the deep integration of the new generation of information technology and industrial manufacturing, the industrial Internet will have an all-round, deep and revolutionary impact on the intelligent development of industrial production. IXPE foamed plastics are a representative large-scale manufacturing, and its demonstration effect will radiate to the entire manufacturing industry. In addition, IXPE foamed plastics are widely used in the manufacturing of products such as automobiles, air conditioners, sports equipment, shoe materials, and luggage, with a potential market scale exceeding 228 billion yuan. However, the domestic production of IXPE faces bottlenecks in production capacity and process technology. Specifically, during the production process, raw materials are drawn by a motor and fed into a primary heating furnace for preheating, then enter a high-temperature heating furnace for further processing, and finally are rolled up. In the high-temperature heating furnace, the position of the sheet is likely to shift. Therefore, it is necessary to continuously detect the position of the sheet. When a shift occurs, the rotation speed of the spreading roller is adjusted in a timely manner for deviation correction to avoid wrinkles in the finished product and maintain smooth production operation.

[0003] The current deviation correction method is as follows: Workers visually monitor at all times. When a deviation occurs, the rotation speed of the spreading roller is adjusted in a timely manner for deviation correction to avoid wrinkles in the finished product and maintain smooth production operation. However, this traditional supervision method has limitations such as low production efficiency and tight human resources.

[0004] Therefore, based on the above considerations, it is necessary to propose a new material automatic deviation correction system that uses sensors to continuously monitor the production status of materials and uses an automatic control device for automatic deviation correction, so as to realize the transformation and upgrading of the intelligent production of IXPE foamed plastics, optimize the human resource structure by "reducing the number of employees", control production costs, and at the same time achieve the transformation from manufacturing to "intelligent" manufacturing by "increasing efficiency", thereby improving labor productivity to expand reproduction. Summary of the Invention

[0005] Aiming at the deficiencies of the above-mentioned existing technologies, the purpose of the present invention is to provide a material automatic deviation correction system and method based on computer vision and feedback control to solve the problems such as low production efficiency and tight human resources existing in the traditional manual material deviation correction method.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A material automatic deviation correction system based on computer vision and feedback control of the present invention includes: a material state acquisition module, a material state data preprocessing module, a material state information calculation module, a multi-stage negative feedback control module, and a central service module;

[0008] The material status acquisition module is used to acquire the production status data of the material;

[0009] The material status data preprocessing module is used to preprocess the acquired production status data of the material, filter out unnecessary information to reduce the calculation amount;

[0010] The material status information calculation module is used to calculate the edge position information of the material (the left edge position and the right edge position of the material), and calculate the width information of the material;

[0011] The multi-level negative feedback control module is used to calculate the deviation correction instruction for the material production at the current moment;

[0012] The central service module is used to receive, store and manage the production status data of the material and the corresponding control commands, and issue an alarm notification when it detects that the material production status deviates from the normal threshold.

[0013] An automatic deviation correction method for materials based on computer vision and feedback control according to the present invention, based on the above system, includes the following steps:

[0014] 1) Real-time acquire the production status data of the material during the material production process;

[0015] 2) Preprocess the production status data of the material, and use the minimized recognition area extraction algorithm to extract the minimized recognition area mROI with the strongest contrast between the material and the background;

[0016] 3) Use the bidirectional edge detection algorithm to identify the minimized recognition area mROI, obtain the left edge position and the right edge position of the material at the current moment, and calculate the width of the material at the current moment;

[0017] 4) According to the multi-level negative feedback control algorithm, calculate the deviation correction instruction for the material production at the current moment.

[0018] Further, the step 1) is specifically: real-time capture the video data of the production status during the material production process through the camera arranged in the material production site; the initial video frame captured by the camera is F s ; for each video frame captured by the camera, detect whether the material exists through the material existence detection method.

[0019] Further, the material existence detection method is specifically:

[0020] 11) For each video frame F captured currently i , calculate the difference frame F d = |F i - F s |;

[0021] 12) For each difference frame F d , calculate its information entropy H(F d ) using a statistical formula;

[0022] 13) When the value of the information entropy H(F d ) is greater than the set threshold T F , it is considered that there is material in the video frame.

[0023] Furthermore, the functional expression of the information entropy in step 12) is:

[0024]

[0025] where F d is the difference frame, h i is the total number of pixel points with value i after graying the difference frame, and p i is the frequency of pixel points with value i among all pixel points.

[0026] Furthermore, the preprocessing of the production status data of the material in step 2) specifically includes:

[0027] 21) Scan the video frame from top to bottom using a window of fixed size, where the height of the window is 50 pixels and the width is the same as the width of the video frame;

[0028] 22) Calculate the horizontal gradient value for each video frame window obtained by each scan;

[0029] 23) Select the window with the largest horizontal gradient value among all windows as the recognition window.

[0030] Furthermore, the minimum recognition area extraction algorithm in step 2) is:

[0031] 24) Obtain the recognition result of the previous frame from the historical recognition data, that is, the left edge position x l (t - 1) and the right edge position x r (t - 1) of the material. If there is no historical recognition data, temporarily skip the minimum recognition area extraction step;

[0032] 25) Set the horizontal detection threshold T x of the material and the maximum movement range x m of the material per frame;

[0033] 26) Extract the minimum recognition area mROI of the current frame. The specific extraction method is: its x-axis range: the left recognition area is (x l (t - 1) - T x - 2x m , x l(t - 1) + 2x m ); The right recognition area is (x r (t - 1) - 2x m , x r (t - 1) + T x + 2x m ); The range of its y-axis: The heights of the left recognition area and the right recognition area are consistent with the height of the recognition window.

[0034] Further, the bidirectional edge detection algorithm in step 3) is specifically as follows:

[0035] 31) If the minimized recognition area mROI has been extracted, use the super-resolution method to upsample the minimized recognition area mROI to obtain a clear production status video frame; otherwise, temporarily skip the upsampling step;

[0036] 32) Use the Canny operator to calculate the edge image of the recognition window or the minimized recognition area mROI;

[0037] 33) Classify all edge points in the edge image using static state edge points and motion state edge points;

[0038] 34) By finding the edge of the background in the horizontal direction and the edge of the material in the vertical direction, and then using a complementary filter to obtain the material edge recognition result.

[0039] Further, the complementary filter in step 34) is specifically as follows:

[0040]

[0041] Among them, x(t) is the position of the material in the x-axis direction at time t, and x(t - 1) is the x-axis position of the material at time t - 1; x h (t) is the position of the material on the x-axis calculated in the horizontal direction at time t, x v (t) is the position of the material on the x-axis calculated in the vertical direction at time t.

[0042] Further, the width of the material at the current moment in step 3) is w(t), and its calculation formula is w(t) = x r (t) - x l (t), where x r (t), x l (t) are the right edge position and the left edge position at the current moment respectively.

[0043] Further, the multi-stage negative feedback control algorithm in step 4) is specifically as follows:

[0044] 41) The function expression of negative feedback control is:

[0045] α * f l (t) - β * f r (t) = γ * (x(t) - x s )

[0046] Among them, f l (t) represents the rotation frequency of the left spreading roller used to control the production position of the material at time t, and f r (t) represents the rotation frequency of the right spreading roller used to control the production position of the material at time t; x s represents the set standard position of the material; α, β, and γ respectively represent the relationship factors between the material position and the rotation frequency of the spreading roller;

[0047] 42) Starting from minimizing the change rate of the spreading roller rotation frequency, the multi-stage negative feedback control algorithm expression is obtained as:

[0048]

[0049]

[0050] Among them, |f l (t) - f l (t - 1)| + |f r (t) - f r (t - 1)| is the numerical value of the change in the rotation frequency of the left and right spreading rollers; by calculating the position difference at the x(t) - x(t - 1) moment, that is, the numerical value of, to determine the change amount of the two spreading rollers, and then by minimizing the numerical value of the change in the rotation frequency of the two spreading rollers, calculate the rotation frequency f of each spreading roller at time t l (t) and f r (t); use the actually observed to correct γ, thereby completing the multi-stage negative feedback control.

[0051] Advantages of the present invention:

[0052] 1. The present invention can effectively detect the edge position and width of the material in the material production environment, and achieve a detection effect at the millimeter level.

[0053] 2. The present invention adopts a minimized recognition area and a two-way tracking algorithm, and can accurately identify the production state of the material in a diverse and complex background industrial production environment; at the same time, due to the adoption of a multi-stage negative feedback automatic control method, the present invention can effectively adapt to the parameter differences of different production line control devices, achieving an efficient and stable automatic deviation correction effect. Description of the Drawings

[0054] Figure 1 is the application scenario schematic diagram of the system of the present invention.

[0055] Figure 2 This is the system method flow chart of the present invention. Detailed implementation manners

[0056] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with embodiments and the accompanying drawings. The content mentioned in the implementation manners does not limit the present invention.

[0057] Refer to Figure 1 As shown, an automatic material rectification system based on computer vision and feedback control of the present invention includes: a material state acquisition module, a material state data preprocessing module, a material state information calculation module, a multi-stage negative feedback control module, and a central service module;

[0058] The material state acquisition module is used to acquire the production state data of the material;

[0059] The material state data preprocessing module is used to preprocess the acquired production state data of the material, and filter out unnecessary information to reduce the calculation amount;

[0060] The material state information calculation module is used to calculate the edge position information of the material (the left edge position and the right edge position of the material), and calculate the width information of the material;

[0061] The multi-stage negative feedback control module is used to calculate the rectification instruction for the current moment of material production (that is, the rotation frequencies of the left and right spreading rollers);

[0062] The central service module is used to receive, store, and manage the production state data of the material and the corresponding control commands, and issue an alarm notification when it detects that the material production state deviates from the normal threshold.

[0063] Refer to Figure 2 As shown, an automatic material rectification method based on computer vision and feedback control of the present invention includes the following steps:

[0064] 1) Real-time acquisition of the production state data of the material during the material production process;

[0065] Specifically, step 1) is specifically: real-time shooting of the video data of the production state during the material production process through a camera arranged at the material production site; the initial video frame collected by the camera is F s ; for each video frame collected by the camera, a material existence detection method is used to detect whether the material exists.

[0066] Among them, the material production site is a vertical high-temperature heating furnace, and the raw material expands in the high-temperature heating furnace to the specified product size; at the same time, there are two left stretching rollers and two right stretching rollers inside the high-temperature heating furnace, and the material passes through the middle of the stretching rollers on both sides and is affected by the frictional force of the stretching rollers to change the production position of the material.

[0067] Among them, the specific method for detecting the existence of the material is as follows:

[0068] 11) For each video frame F captured currently i , calculate the difference frame F d = |F i - F s |;

[0069] 12) For each difference frame F d , use the statistical formula to calculate its information entropy H(F d );

[0070] 13) When the value of the information entropy H(F d ) is greater than the set threshold T F , it is considered that there is material in the video frame.

[0071] More specifically, the function expression of the information entropy in step 12) is:

[0072]

[0073] Among them, F d is the difference frame, h i is the total number of pixel points with the value of i after the difference frame is grayscaled, and p i is the frequency of the pixel points with the value of i appearing among all pixel points.

[0074] 2) Preprocess the production status data of the material, and use the minimum recognition area extraction algorithm to extract the minimum recognition area mROI with the strongest contrast between the material and the background;

[0075] Specifically, the preprocessing of the production status data of the material in step 2) specifically includes:

[0076] 21) Scan the video frame from top to bottom using a window of a fixed size, the height of the window is 50 pixels, and the width is the same as the width of the video frame;

[0077] 22) Calculate the horizontal gradient value of the video frame window obtained by each scan;

[0078] 23) Select the window with the largest horizontal gradient value among all windows as the recognition window.

[0079] Among them, the algorithm for extracting the minimized recognition area in step 2) is as follows:

[0080] 24) Obtain the recognition result of the previous frame from the historical recognition data, that is, the left edge position x l (t - 1) and the right edge position x r (t - 1). If there is no historical recognition data, temporarily skip the step of extracting the minimized recognition area;

[0081] 25) Set the detection threshold T for the horizontal direction of the material x and the maximum movement range x of the material per frame m ;

[0082] 26) Extract the minimized recognition area mROI of the current frame. The specific extraction method is as follows: The x-axis range: for the left recognition area, it is (x l (t - 1) - T x - 2x m , x l (t - 1)+2x m ); for the right recognition area, it is (x r (t - 1) - 2x m , x r (t - 1)+T x +2x m ); The y-axis range: the heights of the left recognition area and the right recognition area are the same as the height of the recognition window.

[0083] 3) Use the bidirectional edge detection algorithm to recognize the minimized recognition area mROI, obtain the left edge position and the right edge position of the material at the current moment, and calculate the width of the material at the current moment;

[0084] Among them, the bidirectional edge detection algorithm in step 3) is specifically as follows:

[0085] 31) If the minimized recognition area mROI has been extracted, use the super-resolution method to upsample the minimized recognition area mROI to obtain a clear video frame of the production state; otherwise, temporarily skip the upsampling step;

[0086] 32) Use the Canny operator to calculate the edge image of the recognition window or the minimized recognition area mROI;

[0087] 33) Classify all the edge points in the edge image using the static state edge points and the moving state edge points;

[0088] 34) By finding the edge of the background in the horizontal direction and the edge of the material in the vertical direction, and then using the complementary filter to obtain the material edge recognition result.

[0089] Specifically, the complementary filter in step 34) is as follows:

[0090]

[0091] Among them, x(t) is the position of the material in the x-axis direction at time t, and x(t - 1) is the position of the material in the x-axis at time t - 1; x h (t) is the position of the material on the x-axis calculated in the horizontal direction at time t, and x v (t) is the position of the material on the x-axis calculated in the vertical direction at time t.

[0092] In the said step 3), the width of the material at the current moment is w(t), and its calculation formula is w(t) = x r (t) - x l (t), where x r (t) and x l (t) are respectively the right edge position and the left edge position at the current moment.

[0093] 4) Calculate the deviation correction instruction for the material production at the current moment according to the multi-stage negative feedback control algorithm;

[0094] Specifically, the multi-stage negative feedback control algorithm in the said step 4) is specifically as follows:

[0095] 41) The function expression of the negative feedback control is:

[0096] α * f l (t) - β * f r (t) = γ * (x(t) - x s )

[0097] Among them, f l (t) represents the rotation frequency of the left spreading roller used to control the production position of the material at time t, and f r (t) represents the rotation frequency of the right spreading roller used to control the production position of the material at time t; x s represents the set standard position of the material; α, β, and γ respectively represent the relationship factors between the material position and the rotation frequency of the spreading roller;

[0098] 42) Starting from minimizing the change rate of the rotation frequency of the spreading roller, the expression of the multi-stage negative feedback control algorithm is obtained as:

[0099]

[0100]

[0101] Among them, |f l (t) - f l (t - 1)| + |f r (t) - f r(t - 1) | is the numerical value of the change in the rotation frequency of the expansion rods on both the left and right sides; by calculating the position difference at the moment of x(t) - x(t - 1), that is the numerical value of, to determine the change amount of the two expansion rods, and then by minimizing the numerical value of the change in the rotation frequency of the expansion rods on both sides, calculate the rotation frequency f of each expansion rod at time t l (t) and f r (t); use the actually observed to correct γ, thereby completing the multi - level negative feedback control.

[0102] There are many specific application ways of the present invention. The above - mentioned is only the preferred implementation manner of the present invention. It should be pointed out that for those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements can still be made, and these improvements should also be regarded as the protection scope of the present invention.

Claims

1. An automatic material rectification method based on computer vision and feedback control, based on an automatic material rectification system based on computer vision and feedback control. The system includes: Material status acquisition module, material status data preprocessing module, material status information calculation module, multi-level negative feedback control module and central service module; The material status acquisition module is used to acquire the production status data of the material; The material status data preprocessing module is used to preprocess the acquired production status data of the material, filter out unnecessary information to reduce the calculation amount; The material status information calculation module is used to calculate the edge position information of the material and calculate the width information of the material; The multi-level negative feedback control module is used to calculate the deviation correction instruction for material production at the current moment; The central service module is used to receive, store and manage the production status data of the material and the corresponding control commands, and issue an alarm notification when it detects that the material production status deviates from the normal threshold; It is characterized in that the method includes the following steps: 1) Real-time acquisition of the production status data of the material during the material production process; 2) Preprocess the production status data of the material, and use the minimized recognition area extraction algorithm to extract the minimized recognition area mROI with the strongest contrast between the material and the background; 3) Use the bidirectional edge detection algorithm to identify the minimized recognition area mROI, obtain the left edge position and right edge position of the material at the current moment, and calculate the width of the material at the current moment; 4) According to the multi-level negative feedback control algorithm, calculate the deviation correction instruction for material production at the current moment; In the step 4), the multi-level negative feedback control algorithm is specifically: 41) The function expression of negative feedback control is: α*f l (t) - β*f r (t) = γ*(x(t) - x s ) Among them, f l (t) represents the rotation frequency of the left spreading roller used to control the production position of the material at time t, and f r (t) represents the rotation frequency of the right spreading roller used to control the production position of the material at time t; x s represents the set standard position of the material; α, β, γ respectively represent the relationship factors between the material position and the spreading roller rotation frequency; 42) Starting from the change rate of the minimized roller rotation frequency, the expression of the multi-level negative feedback control algorithm is obtained: Among them, |f l (t) - f l (t - 1)| + |f r (t) - f r (t - 1)| is the change value of the rotation frequency of the expansion rods on the left and right sides; by calculating the position difference at the moment of x(t) - x(t - 1), that is, the value of, to determine the change amount of the two expansion rods, and then by minimizing the change value of the rotation frequency of the expansion rods on both sides, calculate the rotation frequency f l (t) and f r (t); use the actually observed to correct γ, so as to complete the multi-stage negative feedback control. x(t) is the x-axis position of the material at time t, and x(t - 1) is the x-axis position of the material at time t - 1.

2. The material automatic deviation rectification method based on computer vision and feedback control according to claim 1, characterized in that The specific content of step 1) is as follows: The video data of the production state during the material production process is captured in real time by a camera arranged at the material production site; the initial video frame captured by the camera is F s ; for each video frame captured by the camera, a material presence detection method is used to detect whether the material is present.

3. The automatic material rectification method based on computer vision and feedback control according to claim 2, characterized in that The specific method for detecting the existence of the material is: 11) For each currently captured video frame F i , calculate the difference frame F d = |F i - F s |; 12) For each differential frame F d , calculate its information entropy H(F d ) using a statistical formula; 13) When the information entropy H(F d ) value is greater than the set threshold T F , it is considered that there is material in the video frame.

4. The material automatic deviation rectification method based on computer vision and feedback control according to claim 1, characterized in that In the step 2), the preprocessing of the production status data of the material specifically includes: 21) Use a window with a fixed size to scan the video frame from top to bottom. The height of the window is 50 pixels, and the width is the same as the width of the video frame; 22) For each video frame window obtained by scanning, calculate its horizontal gradient value; 23) Select the window with the largest horizontal gradient value among all windows as the recognition window.

5. The material automatic deviation rectification method based on computer vision and feedback control according to claim 1, characterized in that In the step 2), the minimized recognition area extraction algorithm is: 24) Obtain the recognition result of the previous frame from the historical recognition data, i.e., the left edge position x of the material l (t - 1) and the right edge position x r (t - 1). If there is no historical recognition data, temporarily skip the step of extracting the minimized recognition area; 25) Set the horizontal detection threshold T of the material x and the maximum movement range x of the material per frame m ; 26) Extract the minimum recognition region mROI of the current frame. The specific extraction method is as follows: its x-axis range: for the left recognition region, it is (x l (t - 1) - T x -2x m x l (t - 1) + 2x m ); for the right recognition region, it is (x r (t - 1) - 2x m x r (t - 1) + T x +2x m ); its y-axis range: the heights of the left recognition region and the right recognition region are the same as the height of the recognition window.

6. The automatic material rectification method based on computer vision and feedback control according to claim 1, characterized in that In the step 3), the bidirectional edge detection algorithm is specifically: 31) If the minimized recognition area mROI has been extracted, use the super-resolution method to upsample the minimized recognition area mROI to obtain a clear production status video frame; otherwise, temporarily skip the upsampling step; 32) Use the Canny operator to calculate the edge image of the recognition window or the minimized recognition area mROI; 33) Classify all edge points in the edge image using static state edge points and motion state edge points; 34) By finding the edge of the background in the horizontal direction and the edge of the material in the vertical direction, and then using a complementary filter to obtain the material edge recognition result.

7. The material automatic deviation rectification method based on computer vision and feedback control according to claim 6, wherein In the step 34), the complementary filter is specifically: Among them, x(t) is the x-axis position of the material at time t, and x(t - 1) is the x-axis position of the material at time t - 1; x h (t) is the position of the material on the x-axis at time t calculated in the horizontal direction, x v (t) is the position of the material on the x-axis at time t calculated in the vertical direction.

Images