A video optimization acquisition method for intelligent factory areas based on machine vision

Abstract

The present invention belongs to the technical field of image processing, and particularly relates to a video optimization acquisition method for an intelligent factory area based on machine vision, including: performing Gaussian mixture background modeling on the production line area in the surveillance video, and dividing pixel points into several first categories according to the distances between the Gaussian mixture models of different pixel points; constructing a sub-Gaussian serial number sequence for each pixel point, and dividing the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category; determining the possible abnormal degree of each pixel point in the current video frame according to the dislocation differences between the sub-Gaussian serial number sequences of the pixel points in the second category; and adjusting the acquisition frequency level of the surveillance video according to the magnitudes of the possible abnormal degrees of each pixel point. The present invention dynamically adjusts the sampling frequency level, achieving a balance between monitoring accuracy and system efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to a method for optimizing video acquisition for an intelligent factory area based on machine vision. Background Art

[0002] With the rapid development of intelligent manufacturing technology, the construction of intelligent factory areas has become the core direction of the transformation and upgrading of modern industry. In the process of intelligent management of factory areas, the video monitoring system, as a key information acquisition carrier, undertakes important functions such as production safety monitoring, equipment status analysis, and personnel behavior recognition.

[0003] The setting of the video acquisition frequency directly affects the performance and resource consumption of the system, and needs to be weighed according to the actual application scenario.

[0004] When a high acquisition frequency is adopted, the system can obtain more continuous and detailed dynamic information, which is beneficial to capturing rapidly changing abnormal events (such as equipment failures, personnel's illegal operations, etc.), and improving the real-time performance and accuracy of monitoring. However, high-frequency acquisition will significantly increase the burden of data transmission and storage, resulting in an increase in hardware costs, and at the same time, the occupation of computing resources is also more significant, which may affect the long-term stable operation of the system.

[0005] When a low acquisition frequency is adopted, the data volume can be effectively reduced, and the storage and transmission pressure can be reduced. However, its defect is that transient events may be missed, resulting in monitoring blind spots, especially in the scenarios of high-speed moving targets or sudden events, and the risk of information loss is relatively high.

[0006] Therefore, how to optimize the acquisition frequency on the premise of ensuring the monitoring effect, and balance the data quality and system resource consumption has become an urgent problem to be solved in the video acquisition technology of intelligent factory areas. Summary of the Invention

[0007] To solve the above technical problems that high-frequency acquisition will significantly increase the burden of data transmission and storage, and low-frequency acquisition has a relatively high risk of information loss, the present invention provides a method for optimizing video acquisition for an intelligent factory area based on machine vision, including:

[0008] Set different levels of acquisition frequencies. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest level of acquisition frequency to obtain the production line area in the monitoring video; perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area, and divide the pixel points in the production line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production line area; construct the sub-Gaussian serial number sequences of each pixel point according to the Gaussian mixture models of each pixel point in the production line area and the time-series pixel value sequences of each pixel point, and divide the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of each pixel point in the first category; determine the possible abnormal degree of each pixel point in the current video frame according to the dislocation differences between the sub-Gaussian serial number sequences of each pixel point in the second category; adjust the acquisition frequency level of the monitoring video according to the magnitudes of the possible abnormal degrees of each pixel point.

[0009] Through dynamically adjusting the acquisition frequency of the monitoring video and combining intelligent analysis algorithms, the present invention realizes the efficient operation and accurate anomaly detection of the production line monitoring system. The present invention adopts a multi-level frequency control mechanism. At the initial stage of monitoring, the monitoring video of the production line is comprehensively acquired at the highest acquisition frequency to ensure the integrity of the initial modeling. In the stable operation stage, the sampling frequency is dynamically reduced according to the magnitudes of the possible abnormal degrees of the pixel points, effectively reducing the consumption of data storage and computing resources. The present invention constructs a pixel-level dynamic model through Gaussian mixture background modeling, adopts a two-level classification mechanism, gradually refines the analysis dimension, and captures the time-series characteristics of the production line operation. Based on the differences in the sub-Gaussian serial number sequences, the present invention accurately identifies pixel-level abnormal behaviors through time-series dislocation comparison, dynamically adjusts the sampling frequency level according to the magnitudes of the possible abnormal degrees of the pixel points, realizes the balance between detection accuracy and system efficiency, and can timely discover problems such as equipment state anomalies and product anomalies.

[0010] Preferably, the distances between the Gaussian mixture models of different pixel points satisfy the expression:

[0011] ; in the formula, represents the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area; represents the serial number of the sub-Gaussian model; represents the th pixel point in the production line area, and the distance between the th sub-Gaussian model in the Gaussian mixture model of the th pixel point and the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area; The weight of the th sub-Gaussian model in the Gaussian mixture model of a pixel point, represents the weight of the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area; represents the minimum value function.

[0012] According to the present invention, the weight of the distance between sub-Gaussian models is set according to the weights of the sub-Gaussian models, so that the distance between Gaussian mixture models focuses on the components with more significant weights in the Gaussian mixture models of two pixel points, avoiding the interference of low-weight components on the overall distance calculation, and improving the robustness of distance measurement.

[0013] Preferably, the method for obtaining the distance between sub-Gaussian models is: taking the mean parameter and standard deviation parameter of the sub-Gaussian model as the feature vectors of the sub-Gaussian model, and taking the Euclidean distance between the feature vectors of the corresponding sub-Gaussian models in the Gaussian mixture models of two pixel points as the distance between the corresponding sub-Gaussian models in the Gaussian mixture models of these two pixel points.

[0014] Preferably, the sequence of temporal pixel values is a sequence composed of the pixel values corresponding to each frame of the surveillance video of the pixel point.

[0015] Preferably, the construction of the sequence of sub-Gaussian model numbers of each pixel point includes: for any pixel value in the sequence of temporal pixel values of the pixel point, obtaining the sub-Gaussian model number to which the pixel value belongs, and when the pixel value does not belong to any sub-Gaussian model, marking the sub-Gaussian model number to which the pixel value belongs as the result after adding one to the number of sub-Gaussian models; forming a sequence of the sub-Gaussian model numbers to which all pixel values in the sequence of temporal pixel values of each pixel point belong in the order of the pixel values, as the sequence of sub-Gaussian model numbers of each pixel point.

[0016] Through the construction of the sequence of sub-Gaussian model numbers of pixel points, the present invention realizes the effective modeling and classification of temporal pixel data. When the pixel value does not belong to any existing sub-Gaussian model, the result after adding one to the number of sub-Gaussian models is used as the sub-Gaussian model number to which the pixel value belongs, which not only maintains the integrity of the model but also ensures the traceability of outliers, not only significantly improving the expression ability of data features but also providing a structured feature representation for subsequent temporal analysis.

[0017] Preferably, dividing the pixel points in the first category into several second categories according to the differences between the sub-Gaussian sequence numbers of the pixel points in the first category includes: for any two pixel points in the first category, using the DTW algorithm to match the sub-Gaussian sequence numbers of the two pixel points to obtain the corresponding relationship between the sub-Gaussian model numbers in the sub-Gaussian sequence numbers of the two pixel points; determining the distance between the sub-Gaussian sequence numbers of the two pixel points according to the corresponding relationship; clustering all the pixel points in each first category according to the distances between the sub-Gaussian sequence numbers of the pixel points in each first category, and dividing the pixel points in each first category into multiple second categories.

[0018] Preferably, determining the distance between the sub-Gaussian sequence numbers of the two pixel points includes: dividing the starting data segment and the ending data segment in the sub-Gaussian sequence numbers of each pixel point; using the elements corresponding to the starting data segment and the ending data segment in the sub-Gaussian sequence numbers of another pixel point as the starting matching element and the ending matching element respectively; removing the starting data segment, the ending data segment, the starting matching element and the ending matching element from the sub-Gaussian sequence numbers of the two pixel points; using the DTW distance between the remaining sub-Gaussian model numbers in the sub-Gaussian sequence numbers of the two pixel points as the distance between the sub-Gaussian sequence numbers of the two pixel points.

[0019] By removing potential outliers in the head and tail matching segments, the present invention effectively improves the accuracy of the distance measurement of the core data segment, making the distance measurement more focused on the essential feature changes of the pixel points.

[0020] Preferably, the possible anomaly degree satisfies the expression:

[0021] ; where, the current video frame corresponding time is denoted as T time, represents the possible anomaly degree of the th pixel point in the video frame at T time; represents the possible anomaly degree of the th reference pixel point of the th pixel point in the video frame at time; represents the change misalignment time between the th pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point in the video frame at T time belongs; represents the serial number of the sub-Gaussian model to which the gray value of the th reference pixel point of the th pixel point belongs at The serial number of the sub-Gaussian model to which the gray value in the video frame at a moment belongs; Indicating the number of reference pixel points of the

[0022] Preferably, the method for obtaining the reference pixel points is as follows: For any two pixel points in the second category, when obtaining the distance between the sub-Gaussian serial number sequences of these two pixel points, the result after removing the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian serial number sequences of these two pixel points is used as the corrected sub-Gaussian serial number sequences of the two pixel points; in response to the time corresponding to the first element in the corrected sub-Gaussian serial number sequence of the first pixel point being earlier than the time corresponding to the first element in the corrected sub-Gaussian serial number sequence of the second pixel point, the first pixel point is used as the reference pixel point of the second pixel point, and vice versa, the second pixel point is used as the reference pixel point of the first pixel point; the change misalignment time is the time difference corresponding to the first element in the corrected sub-Gaussian serial number sequences of these two pixel points.

[0023] The present invention performs refined processing on the sub-Gaussian serial number sequences of any two pixel points in the second category. By removing the starting data segment, the ending data segment, and the head and tail matching elements, a corrected sub-Gaussian serial number sequence with higher temporal consistency is obtained, effectively eliminating the interference caused by the sequence boundary effect and the matching deviation, and significantly improving the accuracy of subsequent temporal analysis. The present invention provides an accurate quantitative basis for subsequent spatio-temporal change analysis by calculating the time difference (i.e., the change misalignment time) between the corrected sub-Gaussian serial number sequences of two pixel points.

[0024] Preferably, the adjusting the acquisition frequency level of the surveillance video according to the possible abnormal degree of each pixel point includes: in response to the possible abnormal degree of a pixel point in the current video frame being greater than a preset abnormal threshold, the pixel point is used as an abnormal pixel point; performing connectivity analysis on the abnormal pixel points to obtain a plurality of connected domains, and in response to the number of pixel points included in the connected domain exceeding a preset first number, the connected domain is used as an abnormal area; in response to the existence of an abnormal area in the video frame, the acquisition frequency of the surveillance video is adjusted to the highest level, and the staff is reminded to check whether there is actually an abnormality in the abnormal area of the surveillance video; in response to the non-existence of an abnormal area in M consecutive frames, or the staff confirming that there is no abnormality in the abnormal area, the acquisition frequency of the surveillance video is reduced by one level, where M is a preset second number.

[0025] The beneficial effects of the present invention are as follows: By dynamically adjusting the monitoring video acquisition frequency and combining intelligent analysis algorithms, the present invention realizes the efficient operation and accurate anomaly detection of the production line monitoring system. The present invention adopts a multi-level frequency regulation mechanism. In the initial stage of monitoring, the highest acquisition frequency is used to comprehensively obtain the monitoring videos of the production line to ensure the integrity of the initial modeling. In the stable operation stage, the sampling frequency is dynamically reduced according to the possible anomaly degree of pixel points, effectively reducing the consumption of data storage and computing resources. The present invention constructs a pixel-level dynamic model through Gaussian mixture background modeling, adopts a two-level classification mechanism, gradually refines the analysis dimension, and captures the time series characteristics of the production line operation. Based on the differences in the sub-Gaussian sequence numbers, the present invention accurately identifies pixel-level abnormal behaviors through time series misalignment comparison, dynamically adjusts the sampling frequency level according to the possible anomaly degree of pixel points, achieves the balance between detection accuracy and system efficiency, and can timely discover problems such as abnormal equipment status and product anomalies. Description of the Drawings

[0026] By referring to the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0027] Figure 1 is a flowchart schematically showing a video optimization acquisition method for a smart factory area based on machine vision in the present invention;

[0028] Figure 2 is a schematic diagram showing the matching results of the sub-Gaussian sequence numbers of pixel points P1 and P2;

[0029] Figure 3 is a schematic diagram showing the start data segment, end data segment, start matching element, and end matching element of pixel points P1 and P2;

[0030] Figure 4 is a schematic diagram showing the corresponding relationship between the sub-Gaussian model numbers of pixel points P1 and P2 after removing the start data segment, end data segment, start matching element, and end matching element of pixel points P1 and P2. Detailed Embodiments

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0032] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0033] An embodiment of the present invention discloses a video optimization acquisition method for an intelligent factory area based on machine vision. Referring to Figure 1 , it includes steps S1 - S5:

[0034] S1. Set acquisition frequencies of different levels. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest - level acquisition frequency to obtain the production - line area in the monitoring video.

[0035] Specifically, in the embodiment of the present invention, three levels of acquisition frequencies are set. The higher the level, the greater the acquisition frequency. The acquisition frequency of the first level is 15 FPS, the acquisition frequency of the second level is 30 FPS, and the acquisition frequency of the third level is 60 FPS. In other embodiments, the implementer can set the levels and the acquisition frequencies of each level according to the actual implementation situation.

[0036] At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest - level acquisition frequency. Obtain the production - line area in the monitoring video.

[0037] It should be noted that the method for obtaining the production - line area in the embodiment of the present invention is not limited. Since the production line is stationary and the shooting angle of the camera is stationary, the position of the production line in each frame of the monitoring video is fixed. Therefore, for the production - line area in the monitoring video, it can be calibrated manually or obtained by methods such as semantic segmentation.

[0038] S2. Perform Gaussian mixture background modeling on the production - line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production - line area, and divide the pixel points in the production - line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production - line area.

[0039] Specifically, use the Gaussian mixture model (GMM) technology to perform Gaussian mixture background modeling on the production - line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production - line area. In the embodiment of the present invention, the number of sub - Gaussian models in the Gaussian mixture background modeling technology is set to 5. In other embodiments, the implementer can set the number of sub - Gaussian models according to the actual implementation situation.

[0040] It should be noted that the core idea of Gaussian mixture background modeling is to use multiple Gaussian distributions (i.e., sub-Gaussian models) to describe the color or brightness changes of each pixel point, so as to distinguish the background (stable part) and the foreground (moving target). After Gaussian mixture background modeling, the background is the stable color feature of each pixel point, and the foreground is the sudden change color feature of each pixel point. In a smart factory, if the products on the production line are transported regularly, then the foreground is the environmental background at the position of the pixel point, the color features of the products passing through the position of the pixel point, etc., and the foreground is the foreign objects that suddenly break into the production line, the walking personnel, etc.

[0041] Furthermore, for each pixel point in the production line area, for each sub-Gaussian model in the Gaussian mixture model of each pixel point, the sub-Gaussian models are sorted in descending order of the weights of the sub-Gaussian models.

[0042] The distance between the Gaussian mixture models of different pixel points in the production line area satisfies the expression:

[0043] ;

[0044] In the formula, represents the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area; represents the serial number of the sub-Gaussian model; represents the distance between the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area and the th sub-Gaussian model in the Gaussian mixture model of the th pixel point; represents the weight of the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area, represents the weight of the th sub-Gaussian model in the Gaussian mixture model of the th pixel point; represents the minimum value function.

[0045] In the process of Gaussian mixture background modeling, if a certain type of feature in the temporal pixel value sequence of a pixel point has more corresponding gray values, the weight of the sub-Gaussian model corresponding to this type of feature is greater. Therefore, in the Gaussian mixture model of each pixel point, the sub-Gaussian model with the largest weight usually represents the environmental background feature at the position of this pixel point. Since the environmental background features at the positions of each pixel point may be inconsistent, there may be a large difference in the sub-Gaussian models with the largest weight in the Gaussian mixture models of each pixel point. At the same time, the present invention mainly considers whether the features are similar when the products on the production line reach the positions of each pixel point. Therefore, when calculating the distance between the Gaussian mixture models of different pixel points, the distance between the sub-Gaussian models with the largest weight is not considered. Therefore, the serial number of the sub-Gaussian model takes integer values in the range of [2, 5]. Among them, the temporal pixel value sequence is a sequence composed of the pixel values corresponding to a pixel point in each frame of the monitoring video.

[0046] In the formula, represents the distance and the reference weight. When the weight of the sub-Gaussian model is smaller, it means that the pixel values of this type of feature appear less in the temporal pixel value sequence of the th pixel point. Then, when obtaining the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area, the attention degree to the distance between the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point and the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point is smaller. Similarly, when the weight of the sub-Gaussian model is smaller, it means that the pixel values of this type of feature appear less in the temporal pixel value sequence of the th pixel point. Then, when obtaining the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area, the attention degree to the distance between the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point and the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point is smaller.

[0047] Furthermore, the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area and the In the Gaussian mixture model of a pixel, the distance between the sub-Gaussian models satisfies the expression:

[0048] ;

[0049] wherein, represents the mean parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area, represents the mean parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area; represents the standard deviation parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area; represents the standard deviation parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area.

[0050] According to the distances between the Gaussian mixture models of the respective pixels in the production line area, all the pixels in the production line area are clustered by using a density-based spatial clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), and all the pixels in the production line area are divided into multiple categories, denoted as the first category.

[0051] In the embodiments of the present invention, only DBSCAN clustering is described as an example. In other embodiments, those skilled in the art can select a clustering algorithm according to the actual implementation situation, such as mean shift clustering.

[0052] It should be noted that the present invention clusters the pixels according to the distances between the Gaussian mixture models of the pixels, so that in the first category obtained by clustering, except for the sub-Gaussian model with the largest weight in the Gaussian mixture model of each pixel, the remaining corresponding sub-Gaussian models are basically the same.

[0053] S3. According to the Gaussian mixture models of the respective pixels in the production line area and the time-series pixel value sequences of the respective pixels, construct the sub-Gaussian serial number sequences of the respective pixels, and divide the pixels in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the respective pixels in the first category.

[0054] Specifically, obtain the membership degree of each pixel value in the temporal pixel value sequence of each pixel point in the production line area with respect to each sub-Gaussian model of the mixture Gaussian model of each pixel point:

[0055] ;

[0056] Among them, represents the membership degree of the th pixel value in the temporal pixel value sequence of the th pixel point in the production line area with respect to the th sub-Gaussian model in the mixture Gaussian model of the th pixel point; represents the th pixel value in the temporal pixel value sequence of the th pixel point in the production line area; The standard deviation parameter of the th sub-Gaussian model in the mixture Gaussian model of the th pixel point in the production line area; represents the mean parameter of the th sub-Gaussian model in the mixture Gaussian model of the th pixel point in the production line area; represents the absolute value symbol. When the difference between the th pixel value and the mean parameter is smaller, the membership degree of the th pixel value with respect to the th sub-Gaussian model is higher, and when the difference between the th pixel value and the mean parameter is less than , the membership degree is greater than 1. Otherwise, the membership degree is less than 1.

[0057] For any pixel value in the temporal pixel value sequence of a pixel point, in response to the fact that there is a membership degree greater than 1 among the membership degrees of the pixel value with respect to each sub-Gaussian model of the mixture Gaussian model of the pixel point, take the serial number of the sub-Gaussian model corresponding to the maximum membership degree as the serial number of the sub-Gaussian model to which the pixel value belongs; in response to the fact that the membership degrees of the pixel value with respect to each sub-Gaussian model of the mixture Gaussian model of the pixel point are all less than 1, the pixel value does not belong to any sub-Gaussian model of the mixture Gaussian model of the pixel point, and the pixel point is a foreground, and record the serial number of the sub-Gaussian model to which the pixel value belongs as 6.

[0058] Construct a sequence by arranging the serial numbers of the sub-Gaussian models to which all pixel values in the temporal pixel value sequence of each pixel point belong in the order of the pixel values, as the sub-Gaussian serial number sequence of each pixel point.

[0059] It should be noted that when the product is processed and transferred on the production line, at a previous moment, the product may be at one position on the production line, and at the next moment, the product may be transferred to another position on the production line. The positions where the product is transferred on the production line correspond to different pixel points in the production line area of the monitoring video. Each sub-Gaussian model serial number in the sub-Gaussian serial number sequence of each pixel point reflects different characteristics corresponding to the pixel point at each moment. For example, in the first 10 moments, the sub-Gaussian model serial number of a certain pixel point is 1, and the pixel point presents the environmental background characteristic at this time. In the next 5 moments, the sub-Gaussian model serial number of this pixel point is 3, and the pixel point presents a certain color characteristic of the product transferred to the corresponding position of this pixel point at this time. When the 16th moment arrives and the product is transferred to another position on the production line, the sub-Gaussian model serial number of this pixel point changes. During the transfer process of the product, there is the same change rule in the sub-Gaussian serial number sequences of some pixel points in the same first category. Due to the difference in the time when the product is transferred to the corresponding positions of different pixel points, the change time of the sub-Gaussian serial number sequences of these pixel points has dislocation. Therefore, the present invention considers the change dislocation relationship of the sub-Gaussian serial number sequences of different pixel points, matches the sub-Gaussian model serial number sequences of different pixel points in the same first category, and classifies the pixel points again to obtain several second categories.

[0060] Specifically, for any two pixel points in the first category, the dynamic time warping (DTW) algorithm is used to match the sub-Gaussian serial number sequences of these two pixel points to obtain the corresponding relationship between the sub-Gaussian model serial numbers in the sub-Gaussian serial number sequences of these two pixel points. Figure 2 is a schematic diagram of the matching result. Figure 2 The square points in it represent the sub-Gaussian model serial numbers of pixel point P1, the circular points represent the sub-Gaussian model serial numbers of pixel point P2, the solid line represents the curve formed by the sub-Gaussian model serial numbers of pixel point P1 and the curve formed by the sub-Gaussian model serial numbers of P2, and the dotted line represents the corresponding relationship between the sub-Gaussian model serial numbers of pixel points P1 and P2.

[0061] Specifically, for any one pixel point, the first element in the sub-Gaussian serial number sequence of this pixel point is used as the starting element, and the starting element and the elements that are continuously the same as the starting element after the starting element are divided into a data segment as the starting data segment of this pixel point, where being continuously the same as the starting element means that all the elements between this element and the starting element are the same as the starting element. The last element in the sub-Gaussian serial number sequence of this pixel point is used as the ending element, and the ending element and the elements that are continuously the same as the ending element before the ending element are divided into a data segment as the ending data segment of this pixel point, where being continuously the same as the ending element means that all the elements between this element and the ending element are the same as the ending element.

[0062] For any two pixel points in the first category, all the elements corresponding to the starting data segment of any one of these two pixel points in the sub-Gaussian sequence number sequence of the other pixel point are used as the starting matching elements of the other pixel point, and all the elements corresponding to the ending data segment of any one of these two pixel points in the sub-Gaussian sequence number sequence of the other pixel point are used as the ending matching elements of the other pixel point. The starting data segment, ending data segment, starting matching elements, and ending matching elements in the sub-Gaussian sequence number sequences of these two pixel points are removed from the sub-Gaussian sequence number sequences of these two pixel points. The DTW distance between the remaining sub-Gaussian model sequence numbers in the sub-Gaussian sequence number sequences of these two pixel points is used as the distance between the sub-Gaussian sequence number sequences of these two pixel points. Figure 3 It is a schematic diagram of the starting data segment, ending data segment, starting matching elements, and ending matching elements. Figure 3 Among them, B1 and E1 are the starting data segment and ending data segment of pixel point P1, B2 and E2 are the starting data segment and ending data segment of pixel point P2, the square hollow points are the starting matching elements and ending matching elements of pixel point P1, and the circular hollow points are the starting matching elements and ending matching elements of pixel point P2. Figure 4 It is a schematic diagram of the corresponding relationship between the sub-Gaussian model sequence numbers of pixel points P1 and P2 after removing the starting data segment, ending data segment, starting matching elements, and ending matching elements of pixel points P1 and P2.

[0063] According to the distances between the sub-Gaussian sequence number sequences of each pixel point in each first category, all the pixel points in each first category are clustered using the DBSCAN algorithm, and the pixel points in each first category are divided into multiple categories, denoted as the second category.

[0064] S4. Determine the possible abnormal degree of each pixel point in the current video frame according to the dislocation difference between the sub-Gaussian sequence number sequences of each pixel point in the second category.

[0065] It should be noted that the sub-Gaussian sequence number sequences of the pixel points in the same second category have the same change rule. The present invention obtains the possible abnormal degree of each pixel point according to the change dislocation relationship between the Gaussian model sequence numbers of the pixel points in the same second category.

[0066] Specifically, for any two pixel points in any second category, when calculating the distance between the sub-Gaussian serial numbers sequences of these two pixel points, the result after removing the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian serial numbers sequences of these two pixel points is used as the corrected sub-Gaussian serial numbers sequences of the two pixel points. Obtain the time corresponding to the first element in the corrected sub-Gaussian serial numbers sequences of these two pixel points. In response to the time corresponding to the first element in the corrected sub-Gaussian serial numbers sequences of the first pixel point being earlier than the time corresponding to the first element in the corrected sub-Gaussian serial numbers sequences of the second pixel point, use the first pixel point as the reference pixel point of the second pixel point; conversely, use the second pixel point as the reference pixel point of the first pixel point. Use the time difference corresponding to the first element in the corrected sub-Gaussian serial numbers sequences of these two pixel points as the change misalignment time between these two pixel points.

[0067] Similarly, the reference pixel points of each pixel point, and the change misalignment time between each pixel point and each of its reference pixel points can be obtained.

[0068] Furthermore, obtain the possible anomaly degree of each pixel point in the current video frame:

[0069] ;

[0070] where, denote the moment corresponding to the current video frame as moment T, represents the possible anomaly degree of the th pixel point in the video frame at moment T; represents the possible anomaly degree of the th reference pixel point of the th pixel point in the video frame at moment; represents the change misalignment time between the th pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point in the video frame at moment T belongs; represents the serial number of the sub-Gaussian model to which the gray value of the th reference pixel point of the th pixel point in the video frame at moment belongs; represents the number of reference pixel points of the th pixel point; is the exponential function with the natural constant as the base; is the absolute value symbol.

[0071] The th pixel point and its A reference pixel belongs to the same second category. Pixel points in the same second category have the same variation pattern in the sub-Gaussian sequence number. The th pixel point and its th reference pixel point have a variation misalignment time of . Therefore, the sub-Gaussian model sequence number to which the gray value of the th pixel point belongs in the current video frame is most likely the same as the sub-Gaussian model sequence number to which the gray value of its th reference pixel point belongs in the video frame before time. Therefore, the present invention determines the possible abnormality degree of the th pixel point at the video frame at time T according to the variation relationship between the sub-Gaussian model sequences of the th pixel point and its reference pixel points. When the possible abnormality degree of the th pixel point at the video frame at time T is greater, it indicates that the sub-Gaussian model sequence number to which the gray value of the th pixel point belongs in the video frame at time T does not conform to the variation pattern of the sub-Gaussian sequence numbers of the reference pixel points of the th pixel point. Furthermore, it indicates that an abnormality may have occurred at the position where the th pixel point is located, such as an abnormal stop or stutter of the production line, the appearance of foreign objects other than the product at the position where the th pixel point is located, or the occlusion of personnel at the position where the th pixel point is located, etc.

[0072] In the formula, is the weight of the th reference pixel point. When the possible abnormality degree of the th reference pixel point in the video frame at time is greater, when determining the possible abnormality degree of the th pixel point at the video frame at time T, the sub-Gaussian model sequence number to which the gray value of the th reference pixel point belongs in the video frame at time is less concerned.

[0073] It should be noted that when there is no reference pixel point for the pixel points in the second category, the possible abnormality degree of this pixel point is not calculated.

[0074] S5. Adjust the acquisition frequency level of the monitoring video according to the possible abnormality degree of each pixel point.

[0075] Specifically, in response to the possible abnormality degree of a pixel point in the current video frame being greater than a preset abnormality threshold, the pixel point is regarded as an abnormal pixel point.

[0076] Perform connectivity analysis on abnormal pixel points to obtain multiple connected components, each of which contains only abnormal pixel points. In response to the number of pixel points contained in a connected component exceeding a preset first quantity, the connected component is taken as an abnormal area.

[0077] In response to the existence of an abnormal area in a video frame, adjust the acquisition frequency of the surveillance video to the highest level and generate an abnormal alarm to remind the staff to check whether there is actually an abnormality in the abnormal area of the surveillance video.

[0078] In response to the non-existence of an abnormal area in M consecutive frames, or the staff confirming that there is no abnormality in the abnormal area, reduce the acquisition frequency of the surveillance video by one level, where M is a preset second quantity.

[0079] Implementers can set the abnormality threshold, the first quantity, and the second quantity according to the actual implementation situation. For example, the abnormality threshold is 0.8, the first quantity is 5, and the second quantity is 5. It should be noted that since the smallest serial number in the sub-Gaussian serial number sequence is 1 and the largest is 6, the value range of the possible abnormality degree is [0,5]. Therefore, the set abnormality threshold cannot exceed this range.

Claims

1. A video optimization acquisition method for an intelligent factory area based on machine vision, characterized in that Including: Set different levels of acquisition frequencies. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest level of acquisition frequency to obtain the production line area in the monitoring video; Perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area, and divide the pixel points in the production line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production line area; Construct the sub-Gaussian serial number sequence of each pixel point according to the Gaussian mixture model of each pixel point in the production line area and the sequence of temporal pixel values of each pixel point, and divide the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category; Determine the possible degree of abnormality of each pixel point in the current video frame according to the dislocation difference between the sub-Gaussian serial number sequences of the pixel points in the second category; Adjust the acquisition frequency level of the monitoring video according to the magnitude of the possible degree of abnormality of each pixel point; The distance between the Gaussian mixture models of different pixel points satisfies the expression: ； Wherein, represents the distance between the Gaussian mixture model of the -th pixel point and the -th pixel point in the production line area; represents the serial number of the sub-Gaussian model; represents the distance between the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point and the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area; represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area, represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area; represents the minimum value function.

2. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, wherein, The method for obtaining the distance between the sub-Gaussian models is: Take the mean parameter and standard deviation parameter of the sub-Gaussian model as the feature vector of the sub-Gaussian model, and take the Euclidean distance between the feature vectors of the corresponding sub-Gaussian models in the Gaussian mixture models of two pixel points as the distance between the corresponding sub-Gaussian models in the Gaussian mixture models of these two pixel points.

3. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that, The sequence of temporal pixel values is a sequence composed of the pixel values corresponding to each pixel point in each frame of the monitoring video.

4. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1 or 3, characterized in that The construction of the sub-Gaussian serial number sequence of each pixel point includes: For any pixel value in the sequence of temporal pixel values of a pixel point, obtain the serial number of the sub-Gaussian model to which the pixel value belongs. When the pixel value does not belong to any sub-Gaussian model, mark the serial number of the sub-Gaussian model to which the pixel value belongs as the result after adding one to the number of sub-Gaussian models; Form a sequence by arranging the serial numbers of the sub-Gaussian models to which all pixel values in the sequence of temporal pixel values of each pixel point belong in the order of the pixel values, and use it as the sub-Gaussian serial number sequence of each pixel point.

5. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that, The division of the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category includes: For any two pixel points in the first category, use the DTW algorithm to match the sub-Gaussian serial number sequences of these two pixel points to obtain the corresponding relationship between the serial numbers of the sub-Gaussian models in the sub-Gaussian serial number sequences of these two pixel points; determine the distance between the sub-Gaussian serial number sequences of these two pixel points according to the corresponding relationship; Cluster all pixel points in each first category according to the distances between the sub-Gaussian serial number sequences of each pixel point in each first category, and divide the pixel points in each first category into multiple second categories.

6. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 5, wherein, The determination of the distance between the sub-Gaussian serial number sequences of the two pixel points includes: Divide the start data segment and the end data segment in the sub-Gaussian sequence numbers of each pixel point; use the elements corresponding to the start data segment and the end data segment in the sub-Gaussian sequence numbers of another pixel point as the start matching element and the end matching element respectively; remove the start data segment, the end data segment, the start matching element and the end matching element from the sub-Gaussian sequence numbers of the two pixel points; use the DTW distance between the remaining sub-Gaussian model sequence numbers in the sub-Gaussian sequence numbers of the two pixel points as the distance between the sub-Gaussian sequence numbers of the two pixel points.

7. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 6, characterized in that, The possible abnormal degree satisfies the expression: ； Among them, the moment corresponding to the current video frame is denoted as the T moment. represents the possible abnormal degree of the th pixel point in the video frame at the T moment; represents the possible abnormal degree of the th pixel point's th reference pixel point in the video frame at the moment; represents the change dislocation time between the th pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point in the video frame at the T moment belongs; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point's th reference pixel point in the video frame at the moment belongs; represents the number of reference pixel points of the th pixel point.

8. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 7, characterized in that, The method for obtaining the reference pixel point is: For any two pixel points in the second category, when obtaining the distance between the sub-Gaussian sequence numbers of these two pixel points, use the result after removing the start data segment, the end data segment, the start matching element and the end matching element from the sub-Gaussian sequence numbers of these two pixel points as the corrected sub-Gaussian sequence numbers of the two pixel points; in response to the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the first pixel point being earlier than the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the second pixel point, use the first pixel point as the reference pixel point of the second pixel point, and vice versa, use the second pixel point as the reference pixel point of the first pixel point; the change misalignment time is the time difference corresponding to the first element in the corrected sub-Gaussian sequence numbers of these two pixel points.

9. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that, Adjusting the acquisition frequency level of the monitoring video according to the possible abnormal degree of each pixel point includes: In response to the possible abnormal degree of a pixel point in the current video frame being greater than a preset abnormal threshold, use the pixel point as an abnormal pixel point; perform connectivity analysis on the abnormal pixel points to obtain multiple connected regions, and in response to the number of pixel points included in the connected region exceeding a preset first number, use the connected region as an abnormal region; in response to the existence of an abnormal region in the video frame, adjust the acquisition frequency of the monitoring video to the highest level and remind the staff to check whether there is actually an abnormality in the abnormal region in the monitoring video; in response to the non-existence of an abnormal region in M consecutive frames, or the staff confirms that there is no abnormality in the abnormal region, reduce the acquisition frequency of the monitoring video by one level, where M is a preset second number.

Images