Video optimization acquisition method for smart factory based on machine vision

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a video optimization acquisition method for a smart factory based on machine vision, and the method comprises the steps: carrying out the mixed Gaussian background modeling of a production line region in a monitoring video, and dividing pixel points into a plurality of first types according to the distance between the mixed Gaussian models of different pixel points; constructing a sub Gaussian sequence number sequence of each pixel point, and dividing the pixel points in the first category into a plurality of second categories according to the difference between the sub Gaussian sequence number sequences of each pixel point in the first category; determining the possible abnormal degree of each pixel point in the current video frame according to the dislocation difference between the sub Gaussian sequence number sequences of each pixel point in the second category; and adjusting the acquisition frequency level of the monitoring video according to the possible abnormal degree of each pixel point. According to the invention, the sampling frequency level is dynamically adjusted, and the balance between monitoring precision and system efficiency is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to a method for optimizing video acquisition for an intelligent factory area based on machine vision. Background Art

[0002] With the rapid development of Industry 4.0 and intelligent manufacturing technologies, the construction of intelligent factory areas has become the core direction for the transformation and upgrading of modern industries. In the process of intelligent management of factory areas, the video surveillance system, as a key information acquisition carrier, undertakes important functions such as production safety monitoring, equipment status analysis, and personnel behavior recognition.

[0003] The setting of the video acquisition frequency directly affects the performance and resource consumption of the system, and needs to be weighed according to the actual application scenario.

[0004] When a high acquisition frequency is adopted, the system can obtain more continuous and detailed dynamic information, which is beneficial to capturing rapidly changing abnormal events (such as equipment failures, personnel's illegal operations, etc.), and improving the real-time performance and accuracy of monitoring. However, high-frequency acquisition will significantly increase the burden of data transmission and storage, resulting in an increase in hardware costs, and at the same time, the occupation of computing resources is also more significant, which may affect the long-term stable operation of the system.

[0005] When a low acquisition frequency is adopted, the data volume can be effectively reduced, and the storage and transmission pressure can be reduced. However, its defect is that transient events may be missed, resulting in monitoring blind spots, especially in scenarios of high-speed moving targets or sudden events, and the risk of information loss is relatively high.

[0006] Therefore, how to optimize the acquisition frequency on the premise of ensuring the monitoring effect, and balance the data quality and system resource consumption has become an urgent problem to be solved in the video acquisition technology of intelligent factory areas. Summary of the Invention

[0007] To solve the above technical problems that high-frequency acquisition will significantly increase the burden of data transmission and storage, and low-frequency acquisition has a relatively high risk of information loss, the present invention provides a method for optimizing video acquisition for an intelligent factory area based on machine vision, including: Set different levels of acquisition frequencies. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest level of acquisition frequency to obtain the production line area in the monitoring video. Perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area. Divide the pixel points in the production line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production line area. Construct the sub-Gaussian sequence numbers of each pixel point according to the Gaussian mixture model of each pixel point in the production line area and the sequence of temporal pixel values of each pixel point. Divide the pixel points in the first category into several second categories according to the differences between the sub-Gaussian sequence numbers of each pixel point in the first category. Determine the possible abnormal degree of each pixel point in the current video frame according to the dislocation differences between the sub-Gaussian sequence numbers of each pixel point in the second category. Adjust the acquisition frequency level of the monitoring video according to the magnitudes of the possible abnormal degrees of each pixel point.

[0008] The present invention realizes the efficient operation and precise anomaly detection of the production line monitoring system by dynamically adjusting the acquisition frequency of the monitoring video and combining intelligent analysis algorithms. The present invention adopts a multi-level frequency regulation mechanism. At the initial stage of monitoring, the monitoring video of the production line is comprehensively acquired at the highest acquisition frequency to ensure the integrity of the initial modeling. In the stable operation stage, the sampling frequency is dynamically reduced according to the magnitudes of the possible abnormal degrees of the pixel points, effectively reducing the consumption of data storage and computing resources. The present invention constructs a pixel-level dynamic model through Gaussian mixture background modeling, adopts a two-level classification mechanism, gradually refines the analysis dimension, and captures the time series characteristics of the production line operation. Based on the differences in the sub-Gaussian sequence numbers, the present invention accurately identifies pixel-level abnormal behaviors through temporal dislocation comparison, dynamically adjusts the sampling frequency level according to the magnitudes of the possible abnormal degrees of the pixel points, achieves the balance between detection accuracy and system efficiency, and can timely discover problems such as abnormal equipment states and product anomalies.

[0009] Preferably, the distance between the Gaussian mixture models of different pixel points satisfies the expression: ; where represents the distance between the Gaussian mixture model of the th pixel point and the th pixel point in the production line area; represents the sequence number of the sub-Gaussian model; represents the distance between the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area and the th sub-Gaussian model in the Gaussian mixture model of the th pixel point; represents the th pixel point in the production line area in the Gaussian mixture model of the The weight of the sub-Gaussian model, indicating the weight of the th sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area; indicating the minimum value function.

[0010] According to the present invention, the weight of the distance between sub-Gaussian models is set according to the weights of the sub-Gaussian models, so that the distance between Gaussian mixture models focuses on the components with more significant weights in the Gaussian mixture models of two pixel points, avoiding the interference of low-weight components on the overall distance calculation, and improving the robustness of distance measurement.

[0011] Preferably, the method for obtaining the distance between sub-Gaussian models is: taking the mean parameter and standard deviation parameter of the sub-Gaussian model as the feature vectors of the sub-Gaussian model, and taking the Euclidean distance between the feature vectors of the corresponding sub-Gaussian models in the Gaussian mixture models of two pixel points as the distance between the corresponding sub-Gaussian models in the Gaussian mixture models of these two pixel points.

[0012] Preferably, the sequence of temporal pixel values is a sequence composed of the pixel values corresponding to each frame of the monitoring video of the pixel point.

[0013] Preferably, the construction of the sub-Gaussian serial number sequence of each pixel point includes: for any pixel value in the sequence of temporal pixel values of the pixel point, obtaining the serial number of the sub-Gaussian model to which the pixel value belongs, and when the pixel value does not belong to any sub-Gaussian model, marking the serial number of the sub-Gaussian model to which the pixel value belongs as the result after adding one to the number of sub-Gaussian models; forming a sequence of the serial numbers of the sub-Gaussian models to which all pixel values in the sequence of temporal pixel values of each pixel point belong in the order of the pixel values, as the sub-Gaussian serial number sequence of each pixel point.

[0014] The present invention realizes the effective modeling and classification of temporal pixel data by constructing the sub-Gaussian serial number sequence of pixel points. When the pixel value does not belong to any existing sub-Gaussian model, the result after adding one to the number of sub-Gaussian models is used as the serial number of the sub-Gaussian model to which the pixel value belongs, which not only maintains the integrity of the model but also ensures the traceability of outliers. It not only significantly improves the expression ability of data features but also provides a structured feature representation for subsequent temporal analysis.

[0015] Preferably, dividing the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category includes: for any two pixel points in the first category, using the DTW algorithm to match the sub-Gaussian serial number sequences of these two pixel points to obtain the corresponding relationship between the sub-Gaussian model serial numbers in the sub-Gaussian serial number sequences of these two pixel points; determining the distance between the sub-Gaussian serial number sequences of these two pixel points according to the corresponding relationship; clustering all the pixel points in each first category based on the distances between the sub-Gaussian serial number sequences of the pixel points in each first category, and dividing the pixel points in each first category into multiple second categories.

[0016] Preferably, determining the distance between the sub-Gaussian serial number sequences of the two pixel points includes: dividing the starting data segment and the ending data segment in the sub-Gaussian serial number sequence of each pixel point; using the corresponding elements of the starting data segment and the ending data segment in the sub-Gaussian serial number sequence of another pixel point as the starting matching element and the ending matching element respectively; removing the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian serial number sequences of the two pixel points; taking the DTW distance between the remaining sub-Gaussian model serial numbers in the sub-Gaussian serial number sequences of the two pixel points as the distance between the sub-Gaussian serial number sequences of the two pixel points.

[0017] The present invention effectively improves the accuracy of the distance measurement of the core data segment by removing potential abnormal points in the head and tail matching segments, making the distance measurement more focused on the essential feature changes of the pixel points.

[0018] Preferably, the possible abnormal degree satisfies the expression: ; where the corresponding moment of the current video frame is denoted as the T moment, represents the possible abnormal degree of the th pixel point in the video frame at the T moment; represents the possible abnormal degree of the th reference pixel point of the th pixel point in the video frame at the moment; represents the change misalignment time between the th pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point in the video frame at the T moment belongs; represents the serial number of the sub-Gaussian model to which the gray value of the th reference pixel point of the th pixel point in the video frame at the moment belongs. Indicates the number of reference pixel points of the th pixel point.

[0019] Preferably, the method for obtaining the reference pixel points is as follows: for any two pixel points in the second category, when obtaining the distance between the sub-Gaussian sequence numbers of these two pixel points, the result after removing the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian sequence numbers of these two pixel points is used as the corrected sub-Gaussian sequence numbers of the two pixel points; in response to the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the first pixel point being earlier than the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the second pixel point, the first pixel point is used as the reference pixel point of the second pixel point, and vice versa, the second pixel point is used as the reference pixel point of the first pixel point; the change misalignment time is the time difference corresponding to the first element in the corrected sub-Gaussian sequence numbers of these two pixel points.

[0020] The present invention performs refined processing on the sub-Gaussian sequence numbers of any two pixel points in the second category. By removing the starting data segment, the ending data segment, and the head and tail matching elements, a corrected sub-Gaussian sequence number with higher timing consistency is obtained, effectively eliminating the interference caused by the sequence boundary effect and the matching deviation, and significantly improving the accuracy of subsequent timing analysis. The present invention provides an accurate quantitative basis for subsequent spatio-temporal change analysis by calculating the time difference (i.e., the change misalignment time) between the corrected sub-Gaussian sequence numbers of two pixel points.

[0021] Preferably, adjusting the acquisition frequency level of the surveillance video according to the possible abnormal degree of each pixel point includes: in response to the possible abnormal degree of a pixel point in the current video frame being greater than a preset abnormal threshold, the pixel point is regarded as an abnormal pixel point; performing connectivity analysis on the abnormal pixel points to obtain a plurality of connected domains, and in response to the number of pixel points included in the connected domain exceeding a preset first number, the connected domain is regarded as an abnormal area; in response to the existence of an abnormal area in the video frame, the acquisition frequency of the surveillance video is adjusted to the highest level, and the staff is reminded to check whether there is actually an abnormality in the abnormal area in the surveillance video; in response to the non-existence of an abnormal area in M consecutive frames, or the staff confirms that there is no abnormality in the abnormal area, the acquisition frequency of the surveillance video is reduced by one level, where M is a preset second number.

[0022] The beneficial effects of the present invention are as follows: By dynamically adjusting the monitoring video acquisition frequency and combining intelligent analysis algorithms, the present invention realizes the efficient operation and accurate anomaly detection of the production line monitoring system. The present invention adopts a multi-level frequency regulation mechanism. In the initial stage of monitoring, the highest acquisition frequency is used to comprehensively obtain the monitoring videos of the production line to ensure the integrity of the initial modeling. In the stable operation stage, the sampling frequency is dynamically reduced according to the possible anomaly degree of pixel points, effectively reducing the consumption of data storage and computing resources. The present invention constructs a pixel-level dynamic model through Gaussian mixture background modeling, adopts a two-level classification mechanism, gradually refines the analysis dimension, and captures the time series characteristics of the production line operation. Based on the differences in the sub-Gaussian sequence numbers, the present invention accurately identifies pixel-level abnormal behaviors through time series misalignment comparison, dynamically adjusts the sampling frequency level according to the possible anomaly degree of pixel points, achieves the balance between detection accuracy and system efficiency, and can timely detect problems such as abnormal equipment status and product anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown by way of illustration and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 is a flowchart schematically showing a method for optimizing video acquisition for an intelligent factory area based on machine vision in the present invention; Figure 2 is a schematic diagram showing the matching results of the sub-Gaussian sequence numbers of pixel points P1 and P2; Figure 3 is a schematic diagram showing the start data segment, end data segment, start matching element, and end matching element of pixel points P1 and P2; Figure 4 is a schematic diagram showing the corresponding relationship between the sub-Gaussian model numbers of pixel points P1 and P2 after removing the start data segment, end data segment, start matching element, and end matching element of pixel points P1 and P2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] The following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings.

[0026] An embodiment of the present invention discloses a video optimization acquisition method for an intelligent factory area based on machine vision. Refer to Figure 1 , which includes steps S1 - S5: S1. Set acquisition frequencies of different levels. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest-level acquisition frequency to obtain the production line area in the monitoring video.

[0027] Specifically, in the embodiment of the present invention, three levels of acquisition frequencies are set. The higher the level, the greater the acquisition frequency. The acquisition frequency of the first level is 15 FPS, the acquisition frequency of the second level is 30 FPS, and the acquisition frequency of the third level is 60 FPS. In other embodiments, the implementer can set the levels and the acquisition frequencies of each level according to the actual implementation situation.

[0028] At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest-level acquisition frequency. Obtain the production line area in the monitoring video.

[0029] It should be noted that the embodiment of the present invention does not limit the method for obtaining the production line area. Since the production line is stationary and the shooting angle of the camera is stationary, the position of the production line in each frame of the monitoring video is fixed. Therefore, for the production line area in the monitoring video, it can be calibrated manually or obtained by methods such as semantic segmentation.

[0030] S2. Perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area, and divide the pixel points in the production line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production line area.

[0031] Specifically, use the Gaussian mixture background modeling (Gaussian Mixture Model, GMM) technology to perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area. In the embodiment of the present invention, the number of sub-Gaussian models in the Gaussian mixture background modeling technology is set to 5. In other embodiments, the implementer can set the number of sub-Gaussian models according to the actual implementation situation.

[0032] It should be noted that the core idea of Gaussian mixture background modeling is to use multiple Gaussian distributions (i.e., sub-Gaussian models) to describe the color or brightness changes of each pixel point, so as to distinguish the background (stable part) and the foreground (moving object). After Gaussian mixture background modeling, the background is the stable color feature of each pixel point, and the foreground is the mutated color feature of each pixel point. In a smart factory, if the products on the production line are transported regularly, then the foreground is the environmental background at the location of the pixel point, the color features of the products passing by at the location of the pixel point, etc., and the foreground is the foreign objects suddenly breaking into the production line, the walking personnel, etc.

[0033] Furthermore, for each pixel point in the production line area, for each sub-Gaussian model in the Gaussian mixture model of each pixel point, the sub-Gaussian models are sorted in descending order of the weights of the sub-Gaussian models.

[0034] The distance between the Gaussian mixture models of different pixel points in the production line area satisfies the expression: ; In the formula, represents the distance between the Gaussian mixture models of the -th pixel point and the -th pixel point in the production line area; represents the serial number of the sub-Gaussian model; represents the distance between the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area and the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point; represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area, represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area; represents the minimum value function.

[0035] In the process of Gaussian mixture background modeling, if a certain type of feature in the temporal pixel value sequence of a pixel point has more corresponding gray values, the weight of the sub-Gaussian model corresponding to this type of feature is greater. Therefore, in the Gaussian mixture model of each pixel point, the sub-Gaussian model with the largest weight usually represents the environmental background feature at the position of this pixel point. Since the environmental background features at the positions of each pixel point may be inconsistent, the differences between the sub-Gaussian models with the largest weights in the Gaussian mixture models of each pixel point may be relatively large. At the same time, the present invention mainly considers whether the features are similar when the products on the production line reach the positions of each pixel point. Therefore, when calculating the distance between the Gaussian mixture models of different pixel points, the distance between the sub-Gaussian models with the largest weights is not considered. Therefore, the serial number of the sub-Gaussian model takes integer values in the range of [2, 5]. Wherein, the temporal pixel value sequence is a sequence composed of the pixel values corresponding to the pixel point in each frame of the monitoring video.

[0036] In the formula, represents the distance and the reference weight. When the weight of the sub-Gaussian model is smaller, it indicates that the pixel values of this type of feature appear less in the temporal pixel value sequence of the th pixel point. Then, when obtaining the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area, the attention degree to the distance between the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point and the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point is smaller. Similarly, when the weight of the sub-Gaussian model is smaller, it indicates that the pixel values of this type of feature appear less in the temporal pixel value sequence of the th pixel point. Then, when obtaining the distance between the Gaussian mixture models of the th pixel point and the th pixel point in the production line area, the attention degree to the distance between the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point and the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point is smaller.

[0037] Furthermore, the rd sub-Gaussian model in the Gaussian mixture model of the th pixel point in the production line area and the In the Gaussian mixture model of a pixel, the distance between the sub-Gaussian models satisfies the expression: ; wherein, represents the mean parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area, represents the mean parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area; represents the standard deviation parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area; represents the standard deviation parameter of the sub-Gaussian model in the Gaussian mixture model of the th pixel in the production line area.

[0038] Based on the distances between the Gaussian mixture models of the respective pixels in the production line area, all the pixels in the production line area are clustered using the density-based spatial clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), and all the pixels in the production line area are divided into multiple categories, denoted as the first category.

[0039] In the embodiments of the present invention, only the DBSCAN clustering is described as an example. In other embodiments, those skilled in the art can select a clustering algorithm according to the actual implementation situation, such as mean shift clustering.

[0040] It should be noted that the present invention clusters the pixels according to the distances between the Gaussian mixture models of the pixels, so that in the first category obtained by clustering, for each pixel, except for the sub-Gaussian model with the largest weight in the Gaussian mixture model, the remaining corresponding sub-Gaussian models are basically the same.

[0041] S3. According to the Gaussian mixture models of the respective pixels in the production line area and the temporal pixel value sequences of the respective pixels, construct the sub-Gaussian sequence numbers of the respective pixels, and divide the pixels in the first category into several second categories according to the differences between the sub-Gaussian sequence numbers of the respective pixels in the first category.

[0042] Specifically, obtain the membership degrees of each pixel value in the temporal pixel value sequence of each pixel in the production line area to each sub-Gaussian model of the Gaussian mixture model of each pixel: ; Among them, represents the th pixel value in the time - series pixel value sequence of the th pixel point in the production line area, and is the membership degree of the th sub - Gaussian model in the Gaussian mixture model of the th pixel point; represents the th pixel value in the time - series pixel value sequence of the th pixel point in the production line area; The standard deviation parameter of the th sub - Gaussian model in the Gaussian mixture model of the th pixel point in the production line area; represents the mean parameter of the th sub - Gaussian model in the Gaussian mixture model of the th pixel point in the production line area; represents the absolute - value symbol. When the difference between the th pixel value and the mean parameter is smaller, the membership degree of the th pixel value to the th sub - Gaussian model is higher, and when the difference between the th pixel value and the mean parameter is less than , the membership degree is greater than 1. Conversely, the membership degree is less than 1.

[0043] For any pixel value in the time - series pixel value sequence of a pixel point, in response to the fact that among the membership degrees of this pixel value to each sub - Gaussian model of the Gaussian mixture model of this pixel point, there is a membership degree greater than 1, the serial number of the sub - Gaussian model corresponding to the largest membership degree is used as the serial number of the sub - Gaussian model to which this pixel value belongs; in response to the fact that the membership degrees of this pixel value to each sub - Gaussian model of the Gaussian mixture model of this pixel point are all less than 1, this pixel value does not belong to any sub - Gaussian model of the Gaussian mixture model of this pixel point, this pixel point is a foreground, and the serial number of the sub - Gaussian model to which this pixel value belongs is recorded as 6.

[0044] The serial numbers of the sub - Gaussian models to which all pixel values in the time - series pixel value sequence of each pixel point belong are formed into a sequence in the order of pixel values, serving as the sub - Gaussian serial number sequence of each pixel point.

[0045] It should be noted that when the product is processed and transferred on the production line, at a previous moment, the product may be at one position on the production line, and at the next moment, the product may be transferred to another position on the production line. The positions where the product is transferred on the production line correspond to different pixel points in the production line area of the monitoring video. Each sub-Gaussian model serial number in the sub-Gaussian serial number sequence of each pixel point reflects different characteristics corresponding to the pixel point at each moment. For example, in the first 10 moments, the sub-Gaussian model serial number of a certain pixel point is 1, and the pixel point presents the environmental background characteristic at this time. In the next 5 moments, the sub-Gaussian model serial number of this pixel point is 3, and the pixel point presents a certain color characteristic of the product transferred to the corresponding position of this pixel point at this time. When the 16th moment arrives and the product is transferred to another position on the production line, the sub-Gaussian model serial number of this pixel point changes. During the transfer process of the product, there is the same change rule in the sub-Gaussian serial number sequences of some pixel points in the same first category. Due to the difference in the time when the product is transferred to the corresponding positions of different pixel points, the change time of the sub-Gaussian serial number sequences of these pixel points has dislocation. Therefore, the present invention considers the change dislocation relationship of the sub-Gaussian serial number sequences of different pixel points, matches the sub-Gaussian model serial number sequences of different pixel points in the same first category, and classifies the pixel points again to obtain several second categories.

[0046] Specifically, for any two pixel points in the first category, the dynamic time warping (DTW) algorithm is used to match the sub-Gaussian serial number sequences of these two pixel points to obtain the corresponding relationship between the sub-Gaussian model serial numbers in the sub-Gaussian serial number sequences of these two pixel points. Figure 2 is a schematic diagram of the matching result. Figure 2 In it, the square points represent the sub-Gaussian model serial numbers of pixel point P1, the circular points represent the sub-Gaussian model serial numbers of pixel point P2, the solid line represents the curve formed by the sub-Gaussian model serial numbers of pixel point P1 and the curve formed by the sub-Gaussian model serial numbers of P2, and the dotted line represents the corresponding relationship between the sub-Gaussian model serial numbers of pixel point P1 and P2.

[0047] Specifically, for any one pixel point, the first element in the sub-Gaussian serial number sequence of this pixel point is used as the starting element, and the starting element and the elements that are continuously the same as the starting element after the starting element are divided into a data segment as the starting data segment of this pixel point, where being continuously the same as the starting element means that all the elements between this element and the starting element are the same as the starting element. The last element in the sub-Gaussian serial number sequence of this pixel point is used as the ending element, and the ending element and the elements that are continuously the same as the ending element before the ending element are divided into a data segment as the ending data segment of this pixel point, where being continuously the same as the ending element means that all the elements between this element and the ending element are the same as the ending element.

[0048] For any two pixel points in the first category, all elements corresponding to the starting data segment of any one of these two pixel points in the sub-Gaussian serial number sequence of the other pixel point are used as the starting matching elements of the other pixel point, and all elements corresponding to the ending data segment of any one of these two pixel points in the sub-Gaussian serial number sequence of the other pixel point are used as the ending matching elements of the other pixel point. The starting data segment, ending data segment, starting matching elements, and ending matching elements in the sub-Gaussian serial number sequences of these two pixel points are removed from the sub-Gaussian serial number sequences of these two pixel points. The DTW distance between the remaining sub-Gaussian model serial numbers in the sub-Gaussian serial number sequences of these two pixel points is used as the distance between the sub-Gaussian serial number sequences of these two pixel points. Figure 3 It is a schematic diagram of the starting data segment, ending data segment, starting matching elements, and ending matching elements. Figure 3 In it, B1 and E1 are the starting data segment and ending data segment of pixel point P1, B2 and E2 are the starting data segment and ending data segment of pixel point P2, the square hollow points are the starting matching elements and ending matching elements of pixel point P1, and the circular hollow points are the starting matching elements and ending matching elements of pixel point P2. Figure 4 It is a schematic diagram of the corresponding relationship between the sub-Gaussian model serial numbers of pixel points P1 and P2 after removing the starting data segment, ending data segment, starting matching elements, and ending matching elements of pixel points P1 and P2.

[0049] According to the distances between the sub-Gaussian serial number sequences of each pixel point in each first category, all pixel points in each first category are clustered using the DBSCAN algorithm, and the pixel points in each first category are divided into multiple categories, denoted as the second category.

[0050] S4. Determine the possible abnormal degree of each pixel point in the current video frame according to the dislocation difference between the sub-Gaussian serial number sequences of each pixel point in the second category.

[0051] It should be noted that the sub-Gaussian serial number sequences of pixel points in the same second category have the same change rule. The present invention obtains the possible abnormal degree of each pixel point according to the change dislocation relationship between the Gaussian model serial number sequences of pixel points in the same second category.

[0052] Specifically, for any two pixel points in any second category, when obtaining the distance between the sub-Gaussian serial number sequences of these two pixel points, the result after removing the starting data segment, ending data segment, starting matching element, and ending matching element from the sub-Gaussian serial number sequences of these two pixel points is used as the corrected sub-Gaussian serial number sequences of the two pixel points. Obtain the time corresponding to the first element in the corrected sub-Gaussian serial number sequences of these two pixel points. In response to the time corresponding to the first element in the corrected sub-Gaussian serial number sequence of the first pixel point being earlier than the time corresponding to the first element in the corrected sub-Gaussian serial number sequence of the second pixel point, the first pixel point is used as the reference pixel point of the second pixel point; conversely, the second pixel point is used as the reference pixel point of the first pixel point. The time difference corresponding to the first element in the corrected sub-Gaussian serial number sequences of these two pixel points is used as the change misalignment time between these two pixel points.

[0053] Similarly, the reference pixel points of each pixel point and the change misalignment time between each pixel point and its respective reference pixel points can be obtained.

[0054] Furthermore, obtain the possible anomaly degree of each pixel point in the current video frame: ; where the time corresponding to the current video frame is denoted as time T, represents the possible anomaly degree of the th pixel point in the video frame at time T; represents the possible anomaly degree of the th reference pixel point of the th pixel point in the video frame at time; represents the change misalignment time between the th pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the th pixel point in the video frame at time T belongs; represents the serial number of the sub-Gaussian model to which the gray value of the th reference pixel point of the th pixel point in the video frame at time belongs; represents the number of reference pixel points of the th pixel point; is the exponential function with the natural constant as the base; is the absolute value symbol.

[0055] The th pixel point and its A reference pixel belongs to the same second category. The sub-Gaussian sequence numbers of the pixels in the same second category have the same variation pattern. The th pixel and its th reference pixel have a variation dislocation time of . Therefore, the sub-Gaussian model sequence number to which the gray value of the th pixel belongs in the current video frame is likely to be the same as the sub-Gaussian model sequence number to which the gray value of its th reference pixel belongs in the video frame before moment. Therefore, the present invention determines the possible anomaly degree of the th pixel in the video frame at time T according to the variation relationship of the sub-Gaussian model sequences between the th pixel and its reference pixel. When the possible anomaly degree of the th pixel in the video frame at time T is greater, it indicates that the sub-Gaussian model sequence number to which the gray value of the th pixel belongs in the video frame at time T does not conform to the variation pattern of the sub-Gaussian sequence numbers of the reference pixels of the th pixel. Furthermore, it indicates that an anomaly may have occurred at the position where the th pixel is located, such as an abnormal stop or lag of the production line, the appearance of foreign objects other than the product at the position where the th pixel is located, or the occlusion of personnel at the position where the th pixel is located.

[0056] In the formula, is the weight of the th reference pixel. When the possible anomaly degree of the th reference pixel in the video frame at moment is greater, when determining the possible anomaly degree of the th pixel in the video frame at time T, the sub-Gaussian model sequence number to which the gray value of the th reference pixel belongs in the video frame at moment is less concerned.

[0057] It should be noted that when there is no reference pixel for the pixel in the second category, the possible anomaly degree of this pixel is not calculated.

[0058] S5. Adjust the acquisition frequency level of the monitoring video according to the possible anomaly degrees of the pixels.

[0059] Specifically, in response to the possible anomaly degree of a pixel in the current video frame being greater than a preset anomaly threshold, the pixel is regarded as an abnormal pixel.

[0060] Perform connectivity analysis on abnormal pixel points to obtain multiple connected components, each of which contains only abnormal pixel points. In response to the number of pixel points contained in a connected component exceeding a preset first number, the connected component is regarded as an abnormal area.

[0061] In response to the existence of an abnormal area in the video frame, adjust the acquisition frequency of the surveillance video to the highest level and generate an abnormal alarm to remind the staff to check whether there is actually an abnormality in the abnormal area of the surveillance video.

[0062] In response to the non-existence of an abnormal area in M consecutive frames, or the staff confirms that there is no abnormality in the abnormal area, reduce the acquisition frequency of the surveillance video by one level, where M is a preset second number.

[0063] Implementers can set the abnormality threshold, the first number, and the second number according to the actual implementation situation. For example, the abnormality threshold is 0.8, the first number is 5, and the second number is 5. It should be noted that since the smallest serial number in the sub-Gaussian serial number sequence is 1 and the largest serial number is 6, the value range of the possible abnormality degree is [0, 5]. Therefore, the set abnormality threshold cannot exceed this range.

Claims

1. A video optimization acquisition method for an intelligent factory area based on machine vision, characterized in that, Including: Set different levels of acquisition frequencies. At the initial stage of starting to monitor the production line, acquire the monitoring video at the highest level of acquisition frequency to obtain the production line area in the monitoring video; Perform Gaussian mixture background modeling on the production line area in the monitoring video to obtain the Gaussian mixture model of each pixel point in the production line area, and divide the pixel points in the production line area into several first categories according to the distances between the Gaussian mixture models of different pixel points in the production line area; Construct the sub-Gaussian serial number sequence of each pixel point according to the Gaussian mixture model of each pixel point in the production line area and the sequential pixel value sequence of each pixel point, and divide the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category; Determine the possible degree of abnormality of each pixel point in the current video frame according to the dislocation difference between the sub-Gaussian serial number sequences of the pixel points in the second category; Adjust the acquisition frequency level of the monitoring video according to the magnitude of the possible degree of abnormality of each pixel point.

2. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, wherein The distance between the Gaussian mixture models of different pixel points satisfies the expression: ； In the formula, represents the distance between the Gaussian mixture model of the -th pixel point and the -th pixel point in the production line area; represents the serial number of the sub-Gaussian model; represents the distance between the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point and the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area; represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area, represents the weight of the -th sub-Gaussian model in the Gaussian mixture model of the -th pixel point in the production line area; represents the minimum value function.

3. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 2, wherein The method for obtaining the distance between the sub-Gaussian models is: Take the mean parameter and standard deviation parameter of the sub-Gaussian model as the feature vector of the sub-Gaussian model, and take the Euclidean distance between the feature vectors of the corresponding sub-Gaussian models in the Gaussian mixture models of two pixel points as the distance between the corresponding sub-Gaussian models in the Gaussian mixture models of these two pixel points.

4. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that The sequential pixel value sequence is a sequence composed of the pixel values corresponding to each pixel point in each frame of the monitoring video.

5. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1 or 4, characterized in that, The construction of the sub-Gaussian serial number sequence of each pixel point includes: For any pixel value in the sequential pixel value sequence of a pixel point, obtain the serial number of the sub-Gaussian model to which the pixel value belongs. When the pixel value does not belong to any sub-Gaussian model, mark the serial number of the sub-Gaussian model to which the pixel value belongs as the result after adding one to the number of sub-Gaussian models; Form a sequence of the serial numbers of the sub-Gaussian models to which all pixel values in the sequential pixel value sequence of each pixel point belong in the order of the pixel values as the sub-Gaussian serial number sequence of each pixel point.

6. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that, The division of the pixel points in the first category into several second categories according to the differences between the sub-Gaussian serial number sequences of the pixel points in the first category includes: For any two pixel points in the first category, use the DTW algorithm to match the sub-Gaussian serial number sequences of these two pixel points to obtain the corresponding relationship between the serial numbers of the sub-Gaussian models in the sub-Gaussian serial number sequences of these two pixel points; determine the distance between the sub-Gaussian serial number sequences of these two pixel points according to the corresponding relationship; Cluster all pixel points in each first category according to the distances between the sub-Gaussian serial number sequences of the pixel points in each first category, and divide the pixel points in each first category into multiple second categories.

7. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 6, characterized in that, The determination of the distance between the sub-Gaussian serial number sequences of the two pixel points includes: Divide the starting data segment and the ending data segment in the sub-Gaussian sequence numbers of each pixel; use the elements corresponding to the starting data segment and the ending data segment in the sub-Gaussian sequence numbers of another pixel as the starting matching element and the ending matching element respectively; remove the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian sequence numbers of the two pixels; use the DTW distance between the remaining sub-Gaussian model sequence numbers in the sub-Gaussian sequence numbers of the two pixels as the distance between the sub-Gaussian sequence numbers of the two pixels.

8. A method for optimizing video acquisition in an intelligent factory area based on machine vision according to claim 7, characterized in that, The possible abnormal degree satisfies the expression: ； Among them, the moment corresponding to the current video frame is denoted as the T moment. represents the possible degree of abnormality of the pixel point in the video frame at the T moment; represents the possible degree of abnormality of the th reference pixel point of the pixel point in the video frame at the moment; represents the change misalignment time between the pixel point and its th reference pixel point; represents the serial number of the sub-Gaussian model to which the gray value of the pixel point in the video frame at the T moment belongs; represents the number of reference pixel points of the pixel point.

9. A video optimization acquisition method for an intelligent factory area based on machine vision according to claim 8, characterized in that, The method for obtaining the reference pixel is: For any two pixels in the second category, when obtaining the distance between the sub-Gaussian sequence numbers of these two pixels, use the result after removing the starting data segment, the ending data segment, the starting matching element, and the ending matching element from the sub-Gaussian sequence numbers of these two pixels as the corrected sub-Gaussian sequence numbers of the two pixels; in response to the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the first pixel being earlier than the time corresponding to the first element in the corrected sub-Gaussian sequence numbers of the second pixel, use the first pixel as the reference pixel of the second pixel, and vice versa, use the second pixel as the reference pixel of the first pixel; the change misalignment time is the time difference corresponding to the first element in the corrected sub-Gaussian sequence numbers of these two pixels.

10. The video optimization acquisition method for an intelligent factory area based on machine vision according to claim 1, characterized in that, Adjusting the acquisition frequency level of the monitoring video according to the possible abnormal degree of each pixel includes: In response to the possible abnormal degree of a pixel in the current video frame being greater than a preset abnormal threshold, use the pixel as an abnormal pixel; perform connectivity analysis on the abnormal pixel to obtain multiple connected regions, and in response to the number of pixels included in the connected region exceeding a preset first number, use the connected region as an abnormal region; in response to the existence of an abnormal region in the video frame, adjust the acquisition frequency of the monitoring video to the highest level and remind the staff to check whether there is actually an abnormality in the abnormal region of the monitoring video; in response to the non-existence of an abnormal region in M consecutive frames, or the staff confirms that there is no abnormality in the abnormal region, reduce the acquisition frequency of the monitoring video by one level, where M is a preset second number.

Images