A method and system for optimizing the operation of industrial cameras in complex scenarios

Abstract

This invention discloses a method and system for optimizing the operation of an industrial camera in complex scenes. The method includes: using an industrial camera, dividing the recognition period and acquiring video streams and environmental parameters in the first running phase; extracting keyframes to generate an image set; and forming operating feature parameters by combining environmental parameters with feature extraction and target recognition. The operating feature parameters are then grouped using fuzzy c-means clustering, and the complexity of the recognition period is set. High-complexity time periods are selected and a complex feature set is constructed. A GAN model is used to perform data augmentation simulation on the complex feature set to generate a simulated feature library. In the second running phase, complex scene time periods are detected in real time, and dynamic fusion recognition is performed using the simulated feature library. This invention effectively improves the dynamic recognition rate of industrial cameras in complex scenes through a feature simulation-driven adaptive feature fusion mechanism, enhances the robustness and intelligence of the system, and achieves efficient operation of the industrial camera.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more specifically, to a method and system for optimizing the operation of industrial cameras in complex scenarios. Background Technology

[0002] In industrial automated production, industrial cameras, as core sensor components, directly impact the inspection accuracy and production capacity of the production line. However, in traditional industrial camera vision inspection processes, the parameters of traditional industrial cameras are fixed, the recognition feature analysis methods are relatively simple, and there is a lack of dynamic analysis and dynamic feature amplification processes. This makes it difficult to adapt to complex changes in the industrial vision inspection environment, such as changes in lighting, object motion blur, and multi-target interference. It also lacks the ability to dynamically adjust features in multi-object recognition tasks, resulting in low data utilization efficiency. Furthermore, the algorithms are simplistic, heavily reliant on manual parameter tuning or simple rule engines, and cannot cope with the problem of feature coupling in multiple scenarios. This leads to lower recognition efficiency of industrial cameras, which is detrimental to industrial development. Summary of the Invention

[0003] This invention overcomes the shortcomings of the prior art and proposes a method and system for optimizing the operation of industrial cameras in complex scenarios.

[0004] The first aspect of this invention provides a method for optimizing the operation of industrial cameras in complex scenes, comprising:

[0005] S101: Based on the first running phase of the industrial camera, multiple recognition cycles are set, and in each recognition cycle, video stream data and environmental parameters are acquired;

[0006] S102: Extract keyframes from video stream data to obtain an image set, extract image features and identify targets from the image set, set scene parameters for the image features of the target images, and generate running feature parameters based on the scene parameters and environment parameters;

[0007] S103: Vectorize the running feature parameters for each recognition cycle, and group the features and recognition cycles based on fuzzy c-means clustering. Set the complexity of the recognition cycle based on the distance between the features and the cluster centers.

[0008] S104: Select high-complexity recognition periods as complex scene time periods, extract and store image features from the image set to form a complex feature set, and learn features from the complex feature set based on the GAN generative model to generate simulated features.

[0009] S105: In the second running phase, the complex scene period is identified in real time and marked as the second scene period. The fusion weight is dynamically set according to the similarity of the running feature parameters between the second scene period and the complex scene period. Based on the fusion weight, the real-time recognition features and the simulated features are dynamically fused and recognized to obtain the real-time recognition result.

[0010] In this solution, S101 specifically refers to:

[0011] During the first run of the industrial camera, multiple recognition cycles are set based on the recognition task;

[0012] In each recognition cycle, video streams of the target area are acquired and the target environmental state is identified based on industrial cameras, resulting in video stream data and environmental parameters;

[0013] Environmental parameters include light intensity, object velocity, and background complexity information.

[0014] In this solution, S102,

[0015] For each recognition cycle, keyframes are extracted from the video stream data to obtain an image set;

[0016] The image set is preprocessed by grayscale conversion, smoothing, noise reduction, and normalization.

[0017] The image recognition module identifies and labels the target object to obtain the target region. The number of objects in the target region is then identified to obtain the quantity parameters.

[0018] Based on the Sobel operator, contour feature analysis and contour quantity statistics are performed on the target region to obtain contour parameters;

[0019] In the target area, the color differences in the area are analyzed based on the color histogram, and the color difference degree is evaluated in the form of data variance to obtain the color difference parameter;

[0020] Scene parameters are generated based on quantity parameters, contour parameters, and color difference parameters;

[0021] The scene parameters and environment parameters are integrated to obtain the runtime characteristic parameters.

[0022] In this solution, S103 specifically refers to:

[0023] For each recognition cycle, the corresponding running feature parameters are vectorized to obtain the recognition state vector;

[0024] Each recognition state vector is used as a data point, and the number of data points is set to m.

[0025] Based on the fuzzy c-means clustering algorithm, n cluster centers are set, an n×m membership matrix R is randomly initialized, and the distance from each data point to the cluster center is repeatedly calculated using Euclidean distance. The cluster centers and membership matrix R are updated using the shortest distance until the preset number of iterations is reached.

[0026] Based on the membership matrix R, determine the maximum membership degree of the cluster center to which each data point belongs, and confirm the cluster group of each data point to obtain n cluster groups;

[0027] In n clusters, the classification results based on the recognition state vector are mapped to the running feature parameters and the recognition period to obtain multiple recognition periods;

[0028] For each recognition cycle, calculate the mean distance between the data point corresponding to the running feature parameter and the remaining n-1 cluster centers, and obtain the recognition cycle complexity based on the mean distance.

[0029] In this solution, S104 specifically refers to:

[0030] The complexity is compared with the preset value, and the recognition period with high complexity is selected and marked as the complex scene period.

[0031] Based on all the time periods of the complex scene, keyframes for each time period are extracted from the image set to obtain the complex image set;

[0032] The image recognition module identifies and extracts features of target objects from complex image sets, and extracts the contour features and color features of target objects based on the Sobel operator and color histogram, respectively.

[0033] In a complex image set, a complex scene time period is analyzed, and spatial relationship features are obtained by extracting and storing the spatial relationship information between the target object and the background object.

[0034] Contour features, color features, and spatial relationship features are considered as a complex feature in a recognition cycle;

[0035] A complex feature set is obtained by statistically analyzing all time periods of complex scenarios.

[0036] In this solution, S104 includes:

[0037] A generative model based on GAN is constructed, which includes a generator and a discriminator, and the mean squared error is used as the loss function.

[0038] The complex feature set is imported into the generator as real data, and the complexity of different complex features is imported into the generative model as training weights. The generator then generates simulated training features.

[0039] The simulated training features are imported into the discriminator for probability discrimination, and the generator and discriminator parameters are updated based on the error backpropagation algorithm.

[0040] Repeatedly train the generative model adversarially until the loss function converges to the preset error or the generator and discriminator reach Nash equilibrium.

[0041] Generate a predetermined number of simulated features based on the trained generative model.

[0042] In this solution, S105 specifically refers to:

[0043] During the second runtime phase, complex scene time periods are identified in real time and marked as the second scene time periods;

[0044] In the first runtime period, the most complex scenario period is marked as the first scenario period;

[0045] Based on the similarity of the operational characteristic parameters between the second scene time period and the first scene time period, the fusion weight of the second scene time period is dynamically set;

[0046] Based on industrial cameras, image features of all second scene time periods are acquired in real time and labeled as real-time recognition features;

[0047] The real-time recognition features and simulated features are dynamically fused. The fusion process introduces a linear weighted fusion method and combines fusion weights to perform feature vectorization and feature fusion of the real-time recognition features and simulated features. Based on the fused features, the target object is identified in real time to obtain the real-time recognition result.

[0048] A second aspect of the present invention also provides an industrial camera operation optimization system based on complex scenes. The system includes a memory and a processor. The memory includes an industrial camera operation optimization program based on complex scenes. When the processor executes the industrial camera operation optimization program based on complex scenes, it performs the following steps:

[0049] S101: Based on the first running phase of the industrial camera, multiple recognition cycles are set, and in each recognition cycle, video stream data and environmental parameters are acquired;

[0050] S102: Extract keyframes from video stream data to obtain an image set, extract image features and identify targets from the image set, set scene parameters for the image features of the target images, and generate running feature parameters based on the scene parameters and environment parameters;

[0051] S103: Vectorize the running feature parameters for each recognition cycle, and group the features and recognition cycles based on fuzzy c-means clustering. Set the complexity of the recognition cycle based on the distance between the features and the cluster centers.

[0052] S104: Select high-complexity recognition periods as complex scene time periods, extract and store image features from the image set to form a complex feature set, and learn features from the complex feature set based on the GAN generative model to generate simulated features.

[0053] S105: In the second running phase, the complex scene period is identified in real time and marked as the second scene period. The fusion weight is dynamically set according to the similarity of the running feature parameters between the second scene period and the complex scene period. Based on the fusion weight, the real-time recognition features and the simulated features are dynamically fused and recognized to obtain the real-time recognition result.

[0054] A third aspect of the present invention also provides a computer-readable storage medium comprising an industrial camera operation optimization program based on a complex scene, wherein when the industrial camera operation optimization program based on a complex scene is executed by a processor, the steps of the industrial camera operation optimization method based on a complex scene as described in any of the preceding claims are implemented.

[0055] This invention discloses a method and system for optimizing the operation of an industrial camera in complex scenes. The method includes: using an industrial camera, dividing the recognition period and acquiring video streams and environmental parameters in the first running phase; extracting keyframes to generate an image set; and forming operating feature parameters by combining the environmental parameters with feature extraction and target recognition. The operating feature parameters are then grouped using fuzzy c-means clustering, and the complexity of the recognition period is set. High-complexity time periods are selected and a complex feature set is constructed. A GAN model is used to perform data augmentation simulation on the complex feature set to generate a simulated feature library. In the second running phase, complex scene time periods are detected in real time, and dynamic fusion recognition is performed using the simulated feature library. This invention effectively improves the dynamic recognition rate of industrial cameras in complex scenes through a feature simulation-driven adaptive feature fusion mechanism, enhances the robustness and intelligence of the system, and achieves efficient operation of the industrial camera. Attached Figure Description

[0056] Figure 1 A flowchart of an industrial camera operation optimization method based on complex scenarios according to the present invention is shown;

[0057] Figure 2 A block diagram of an industrial camera operation optimization system based on complex scenes according to the present invention is shown. Detailed Implementation

[0058] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0059] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0060] Figure 1 A flowchart of an industrial camera operation optimization method based on complex scenarios according to the present invention is shown.

[0061] like Figure 1 As shown, the first aspect of the present invention provides a method for optimizing the operation of an industrial camera in a complex scene, comprising:

[0062] S101: Based on the first running phase of the industrial camera, multiple recognition cycles are set, and in each recognition cycle, video stream data and environmental parameters are acquired;

[0063] S102: Extract keyframes from video stream data to obtain an image set, extract image features and identify targets from the image set, set scene parameters for the image features of the target images, and generate running feature parameters based on the scene parameters and environment parameters;

[0064] S103: Vectorize the running feature parameters for each recognition cycle, and group the features and recognition cycles based on fuzzy c-means clustering. Set the complexity of the recognition cycle based on the distance between the features and the cluster centers.

[0065] S104: Select high-complexity recognition periods as complex scene time periods, extract and store image features from the image set to form a complex feature set, and learn features from the complex feature set based on the GAN generative model to generate simulated features.

[0066] S105: In the second running phase, the complex scene period is identified in real time and marked as the second scene period. The fusion weight is dynamically set according to the similarity of the running feature parameters between the second scene period and the complex scene period. Based on the fusion weight, the real-time recognition features and the simulated features are dynamically fused and recognized to obtain the real-time recognition result.

[0067] According to an embodiment of the present invention, step S101 specifically includes:

[0068] During the first run of the industrial camera, multiple recognition cycles are set based on the recognition task;

[0069] In each recognition cycle, video streams of the target area are acquired and the target environmental state is identified based on industrial cameras, resulting in video stream data and environmental parameters;

[0070] Environmental parameters include light intensity, object velocity, and background complexity information.

[0071] It should be noted that multiple recognition cycles are consecutive cycles, and each recognition cycle generally corresponds to one or more tasks, specifically based on recognition requirements. Each recognition cycle corresponds to one or more specific objects to be recognized. Periodic analysis is used to determine the complexity of the scene and working conditions in different recognition periods, thereby selecting features for subsequent analysis. Background complexity information generally includes the categories and quantities of objects other than the target object in the target recognition region that affect recognition (i.e., background objects), which have a certain impact on the recognition rate.

[0072] According to an embodiment of the present invention, S102,

[0073] For each recognition cycle, keyframes are extracted from the video stream data to obtain an image set;

[0074] The image set is preprocessed by grayscale conversion, smoothing, noise reduction, and normalization.

[0075] The image recognition module identifies and labels the target object to obtain the target region. The number of objects in the target region is then identified to obtain the quantity parameters.

[0076] Based on the Sobel operator, contour feature analysis and contour quantity statistics are performed on the target region to obtain contour parameters;

[0077] In the target area, the color differences in the area are analyzed based on the color histogram, and the color difference degree is evaluated in the form of data variance to obtain the color difference parameter;

[0078] Scene parameters are generated based on quantity parameters, contour parameters, and color difference parameters;

[0079] The scene parameters and environment parameters are integrated to obtain the runtime characteristic parameters.

[0080] It should be noted that the image recognition module generally performs target detection based on algorithms such as YOLOv5. Scene parameters include various parameters. Operating feature parameters effectively reflect the complexity of image features and operating conditions in the recognized image, and are dynamically recorded based on parameter values. Each recognition cycle corresponds to one operating feature parameter.

[0081] According to an embodiment of the present invention, step S103 specifically includes:

[0082] For each recognition cycle, the corresponding running feature parameters are vectorized to obtain the recognition state vector;

[0083] Each recognition state vector is used as a data point, and the number of data points is set to m.

[0084] Based on the fuzzy c-means clustering algorithm, n cluster centers are set, an n×m membership matrix R is randomly initialized, and the distance from each data point to the cluster center is repeatedly calculated using Euclidean distance. The cluster centers and membership matrix R are updated using the shortest distance until the preset number of iterations is reached.

[0085] Based on the membership matrix R, determine the maximum membership degree of the cluster center to which each data point belongs, and confirm the cluster group of each data point to obtain n cluster groups;

[0086] In n clusters, the classification results based on the recognition state vector are mapped to the running feature parameters and the recognition period to obtain multiple recognition periods;

[0087] For each recognition cycle, calculate the mean distance between the data point corresponding to the running feature parameter and the remaining n-1 cluster centers, and obtain the recognition cycle complexity based on the mean distance.

[0088] It should be noted that in the n×m membership matrix R, each value Rij represents the membership degree of the j-th data point to the i-th cluster group, where Rij represents the data in the i-th row and j-th column of the matrix. In the fuzzy c-means clustering algorithm, distance analysis can be calculated based on Mahalanobis distance. m is also the number of recognition cycles; each recognition cycle group includes at least one recognition cycle, and the data points corresponding to the recognition cycles in this group all belong to a cluster center, i.e., a cluster group. In calculating the complexity of a recognition cycle, the distance to the other group centers (n-1 cluster centers) is calculated based on the data points with the corresponding running feature parameters. In the clustering group, specifically for each cluster center, the running feature parameters of different recognition cycles can be clustered in the form of data points. In the cluster group, the farther the data point is from other centers, the higher its data dispersion and the greater the difference in recognition state compared to other cycles. Therefore, the complexity assigned to it is also higher. The complexity is proportional to the mean distance.

[0089] According to an embodiment of the present invention, step S104 specifically includes:

[0090] The complexity is compared with the preset value, and the recognition period with high complexity is selected and marked as the complex scene period.

[0091] Based on all the time periods of the complex scene, keyframes for each time period are extracted from the image set to obtain the complex image set;

[0092] The image recognition module identifies and extracts features of target objects from complex image sets, and extracts the contour features and color features of target objects based on the Sobel operator and color histogram, respectively.

[0093] In a complex image set, a complex scene time period is analyzed, and spatial relationship features are obtained by extracting and storing the spatial relationship information between the target object and the background object.

[0094] Contour features, color features, and spatial relationship features are considered as a complex feature in a recognition cycle;

[0095] A complex feature set is obtained by statistically analyzing all time periods of complex scenarios.

[0096] It should be noted that the recognition period with a complexity exceeding the preset value is considered as the complex scene period.

[0097] Complex features involve the integration of multiple features. A complex scene time period corresponds to a specific recognition cycle.

[0098] According to an embodiment of the present invention, S104 includes:

[0099] A generative model based on GAN is constructed, which includes a generator and a discriminator, and the mean squared error is used as the loss function.

[0100] The complex feature set is imported into the generator as real data, and the complexity of different complex features is imported into the generative model as training weights. The generator then generates simulated training features.

[0101] The simulated training features are imported into the discriminator for probability discrimination, and the generator and discriminator parameters are updated based on the error backpropagation algorithm.

[0102] Repeatedly train the generative model adversarially until the loss function converges to the preset error or the generator and discriminator reach Nash equilibrium.

[0103] Generate a predetermined number of simulated features based on the trained generative model.

[0104] It should be noted that the training weights can be used to determine the true probability of the final simulated feature. Each recognition period corresponds to a complex feature, and different complex features also correspond to a level of complexity.

[0105] In this invention, the simulated features under complex working conditions are learned through the GAN model, which effectively achieves the goal of dynamically improving the feature recognition rate. The simulated features are learned and simulated based on the feature set under highly complex industrial environments, thereby improving the dynamic recognition rate under complex scenarios.

[0106] For example, in real-time industrial camera recognition under conditions of uneven lighting and vibration, existing technologies struggle to achieve precise industrial recognition and improve dynamic recognition rates in complex scenarios, thus hindering the efficiency of industrial vision recognition tasks. This invention effectively solves these problems.

[0107] According to an embodiment of the present invention, step S105 specifically includes:

[0108] During the second runtime phase, complex scene time periods are identified in real time and marked as the second scene time periods;

[0109] In the first runtime period, the most complex scenario period is marked as the first scenario period;

[0110] Based on the similarity of the operational characteristic parameters between the second scene time period and the first scene time period, the fusion weight of the second scene time period is dynamically set;

[0111] Based on industrial cameras, image features of all second scene time periods are acquired in real time and labeled as real-time recognition features;

[0112] The real-time recognition features and simulated features are dynamically fused. The fusion process introduces a linear weighted fusion method and combines fusion weights to perform feature vectorization and feature fusion of the real-time recognition features and simulated features. Based on the fused features, the target object is identified in real time to obtain the real-time recognition result.

[0113] It should be noted that the second scenario time period includes multiple periods (i.e., there are multiple complex scenario time periods in the second runtime period), thus resulting in multiple fusion weights.

[0114] It is worth mentioning that in the traditional industrial camera vision inspection process, the parameters of traditional industrial cameras are fixed, the recognition feature analysis methods are relatively simple, and there is a lack of dynamic analysis and dynamic feature amplification processes. It is difficult to adapt to complex changes in the industrial vision inspection environment, such as changes in lighting, object motion blur, multi-target interference and other complex scenarios. It also lacks the ability to dynamically adjust in multi-object recognition tasks (such as multi-object cyclic recognition and classification tasks on conveyor belts), which leads to low recognition efficiency of industrial cameras and is not conducive to industrial development.

[0115] In particular, for more complex recognition periods and more recognition tasks, the number of recognition cycles is often large, and the types and number of target objects are large. As the industrial camera operates, the resulting real-time recognition error gradually increases, and the real-time recognition rate gradually decreases. At this time, manual intervention is often required to adjust the camera parameters to achieve optimization and efficiency, but the practicality of the operation process is not high.

[0116] This invention periodically analyzes video stream data collected by industrial cameras, performs complexity analysis on different recognition scenarios using both scene and environmental parameters, calculates and sets parameters using a clustering model, and identifies recognition cycles with complex conditions and feature learning requirements in real time. After selecting complex scene periods, features are extracted and simulated using a GAN model to achieve dynamic simulation and amplification of features during the recognition process. Finally, the simulated features are dynamically fused with real-time features of different complexities and weights. This allows for dynamic adjustment of recognition features across multiple industrial camera runtimes, continuously optimizing the features and dynamically improving the recognition rate. It is suitable for industrial camera recognition tasks with high complexity conditions.

[0117] According to an embodiment of the present invention, it further includes:

[0118] Based on each runtime segment, the generated simulation features are time-segmented and stored in an extended database;

[0119] The average recognition rate of the industrial camera in the current running segment is determined in real time. If the average recognition rate is lower than the preset recognition rate, the simulated features of the most recent N running segments are extracted to form a set of simulated recognition features.

[0120] The simulated feature set is analyzed for data difference using density clustering. Based on the clustering results, dense and non-dense regions are marked. Data in non-dense regions is filtered and removed as noise data. The simulated feature set is then updated and re-imported into the extended database for coverage.

[0121] It should be noted that density clustering can use algorithms such as DBSCAN to perform cluster analysis on the dataset. The clustering process is mainly used to filter out noisy clusters, that is, feature data that deviates from the overall center point, and mark them as noisy data for removal. N is a preset integer.

[0122] Figure 2 A block diagram of an industrial camera operation optimization system based on complex scenes according to the present invention is shown.

[0123] A second aspect of the present invention also provides an industrial camera operation optimization system 2 based on complex scenes. The system includes a memory 21 and a processor 22. The memory 21 includes an industrial camera operation optimization program based on complex scenes. When the processor 22 executes the industrial camera operation optimization program based on complex scenes, it performs the following steps:

[0124] S101: Based on the first running phase of the industrial camera, multiple recognition cycles are set, and in each recognition cycle, video stream data and environmental parameters are acquired;

[0125] S102: Extract keyframes from video stream data to obtain an image set, extract image features and identify targets from the image set, set scene parameters for the image features of the target images, and generate running feature parameters based on the scene parameters and environment parameters;

[0126] S103: Vectorize the running feature parameters for each recognition cycle, and group the features and recognition cycles based on fuzzy c-means clustering. Set the complexity of the recognition cycle based on the distance between the features and the cluster centers.

[0127] S104: Select high-complexity recognition periods as complex scene time periods, extract and store image features from the image set to form a complex feature set, and learn features from the complex feature set based on the GAN generative model to generate simulated features.

[0128] S105: In the second running phase, the complex scene period is identified in real time and marked as the second scene period. The fusion weight is dynamically set according to the similarity of the running feature parameters between the second scene period and the complex scene period. Based on the fusion weight, the real-time recognition features and the simulated features are dynamically fused and recognized to obtain the real-time recognition result.

[0129] According to an embodiment of the present invention, step S101 specifically includes:

[0130] During the first run of the industrial camera, multiple recognition cycles are set based on the recognition task;

[0131] In each recognition cycle, video streams of the target area are acquired and the target environmental state is identified based on industrial cameras, resulting in video stream data and environmental parameters;

[0132] Environmental parameters include light intensity, object velocity, and background complexity information.

[0133] It should be noted that multiple recognition cycles are consecutive cycles, and each recognition cycle generally corresponds to one or more tasks, specifically based on recognition requirements. Each recognition cycle corresponds to one or more specific objects to be recognized. Periodic analysis is used to determine the complexity of the scene and working conditions in different recognition periods, thereby selecting features for subsequent analysis. Background complexity information generally includes the categories and quantities of objects other than the target object in the target recognition region that affect recognition (i.e., background objects), which have a certain impact on the recognition rate.

[0134] According to an embodiment of the present invention, S102,

[0135] For each recognition cycle, keyframes are extracted from the video stream data to obtain an image set;

[0136] The image set is preprocessed by grayscale conversion, smoothing, noise reduction, and normalization.

[0137] The image recognition module identifies and labels the target object to obtain the target region. The number of objects in the target region is then identified to obtain the quantity parameters.

[0138] Based on the Sobel operator, contour feature analysis and contour quantity statistics are performed on the target region to obtain contour parameters;

[0139] In the target area, the color differences in the area are analyzed based on the color histogram, and the color difference degree is evaluated in the form of data variance to obtain the color difference parameter;

[0140] Scene parameters are generated based on quantity parameters, contour parameters, and color difference parameters;

[0141] The scene parameters and environment parameters are integrated to obtain the runtime characteristic parameters.

[0142] It should be noted that the image recognition module generally performs target detection based on algorithms such as YOLOv5. Scene parameters include various parameters. Operating feature parameters effectively reflect the complexity of image features and operating conditions in the recognized image, and are dynamically recorded based on parameter values. Each recognition cycle corresponds to one operating feature parameter.

[0143] According to an embodiment of the present invention, step S103 specifically includes:

[0144] For each recognition cycle, the corresponding running feature parameters are vectorized to obtain the recognition state vector;

[0145] Each recognition state vector is used as a data point, and the number of data points is set to m.

[0146] Based on the fuzzy c-means clustering algorithm, n cluster centers are set, an n×m membership matrix R is randomly initialized, and the distance from each data point to the cluster center is repeatedly calculated using Euclidean distance. The cluster centers and membership matrix R are updated using the shortest distance until the preset number of iterations is reached.

[0147] Based on the membership matrix R, determine the maximum membership degree of the cluster center to which each data point belongs, and confirm the cluster group of each data point to obtain n cluster groups;

[0148] In n clusters, the classification results based on the recognition state vector are mapped to the running feature parameters and the recognition period to obtain multiple recognition periods;

[0149] For each recognition cycle, calculate the mean distance between the data point corresponding to the running feature parameter and the remaining n-1 cluster centers, and obtain the recognition cycle complexity based on the mean distance.

[0150] It should be noted that in the n×m membership matrix R, each value Rij represents the membership degree of the j-th data point to the i-th cluster group, where Rij represents the data in the i-th row and j-th column of the matrix. In the fuzzy c-means clustering algorithm, distance analysis can be calculated based on Mahalanobis distance. m is also the number of recognition cycles; each recognition cycle group includes at least one recognition cycle, and the data points corresponding to the recognition cycles in this group all belong to a cluster center, i.e., a cluster group. In calculating the complexity of a recognition cycle, the distance to the other group centers (n-1 cluster centers) is calculated based on the data points with the corresponding running feature parameters. In the clustering group, specifically for each cluster center, the running feature parameters of different recognition cycles can be clustered in the form of data points. In the cluster group, the farther the data point is from other centers, the higher its data dispersion and the greater the difference in recognition state compared to other cycles. Therefore, the complexity assigned to it is also higher. The complexity is proportional to the mean distance.

[0151] According to an embodiment of the present invention, step S104 specifically includes:

[0152] The complexity is compared with the preset value, and the recognition period with high complexity is selected and marked as the complex scene period.

[0153] Based on all the time periods of the complex scene, keyframes for each time period are extracted from the image set to obtain the complex image set;

[0154] The image recognition module identifies and extracts features of target objects from complex image sets, and extracts the contour features and color features of target objects based on the Sobel operator and color histogram, respectively.

[0155] In a complex image set, a complex scene time period is analyzed, and spatial relationship features are obtained by extracting and storing the spatial relationship information between the target object and the background object.

[0156] Contour features, color features, and spatial relationship features are considered as a complex feature in a recognition cycle;

[0157] A complex feature set is obtained by statistically analyzing all time periods of complex scenarios.

[0158] It should be noted that the recognition period with a complexity exceeding the preset value is considered as the complex scene period.

[0159] Complex features involve the integration of multiple features. A complex scene time period corresponds to a specific recognition cycle.

[0160] According to an embodiment of the present invention, S104 includes:

[0161] A generative model based on GAN is constructed, which includes a generator and a discriminator, and the mean squared error is used as the loss function.

[0162] The complex feature set is imported into the generator as real data, and the complexity of different complex features is imported into the generative model as training weights. The generator then generates simulated training features.

[0163] The simulated training features are imported into the discriminator for probability discrimination, and the generator and discriminator parameters are updated based on the error backpropagation algorithm.

[0164] Repeatedly train the generative model adversarially until the loss function converges to the preset error or the generator and discriminator reach Nash equilibrium.

[0165] Generate a predetermined number of simulated features based on the trained generative model.

[0166] It should be noted that the training weights can be used to determine the true probability of the final simulated feature. Each recognition period corresponds to a complex feature, and different complex features also correspond to a level of complexity.

[0167] In this invention, the simulated features under complex working conditions are learned through the GAN model, which effectively achieves the goal of dynamically improving the feature recognition rate. The simulated features are learned and simulated based on the feature set under highly complex industrial environments, thereby improving the dynamic recognition rate under complex scenarios.

[0168] For example, in real-time industrial camera recognition under conditions of uneven lighting and vibration, existing technologies struggle to achieve precise industrial recognition and improve dynamic recognition rates in complex scenarios, thus hindering the efficiency of industrial vision recognition tasks. This invention effectively solves these problems.

[0169] According to an embodiment of the present invention, step S105 specifically includes:

[0170] During the second runtime phase, complex scene time periods are identified in real time and marked as the second scene time periods;

[0171] In the first runtime period, the most complex scenario period is marked as the first scenario period;

[0172] Based on the similarity of the operational characteristic parameters between the second scene time period and the first scene time period, the fusion weight of the second scene time period is dynamically set;

[0173] Based on industrial cameras, image features of all second scene time periods are acquired in real time and labeled as real-time recognition features;

[0174] The real-time recognition features and simulated features are dynamically fused. The fusion process introduces a linear weighted fusion method and combines fusion weights to perform feature vectorization and feature fusion of the real-time recognition features and simulated features. Based on the fused features, the target object is identified in real time to obtain the real-time recognition result.

[0175] It should be noted that the second scenario time period includes multiple periods (i.e., there are multiple complex scenario time periods in the second runtime period), thus resulting in multiple fusion weights.

[0176] It is worth mentioning that in the traditional industrial camera vision inspection process, the parameters of traditional industrial cameras are fixed, the recognition feature analysis methods are relatively simple, and there is a lack of dynamic analysis and dynamic feature amplification processes. It is difficult to adapt to complex changes in the industrial vision inspection environment, such as changes in lighting, object motion blur, multi-target interference and other complex scenarios. It also lacks the ability to dynamically adjust in multi-object recognition tasks (such as multi-object cyclic recognition and classification tasks on conveyor belts), which leads to low recognition efficiency of industrial cameras and is not conducive to industrial development.

[0177] In particular, for more complex recognition periods and more recognition tasks, the number of recognition cycles is often large, and the types and number of target objects are large. As the industrial camera operates, the resulting real-time recognition error gradually increases, and the real-time recognition rate gradually decreases. At this time, manual intervention is often required to adjust the camera parameters to achieve optimization and efficiency, but the practicality of the operation process is not high.

[0178] This invention periodically analyzes video stream data collected by industrial cameras, performs complexity analysis on different recognition scenarios using both scene and environmental parameters, calculates and sets parameters using a clustering model, and identifies recognition cycles with complex conditions and feature learning requirements in real time. After selecting complex scene periods, features are extracted and simulated using a GAN model to achieve dynamic simulation and amplification of features during the recognition process. Finally, the simulated features are dynamically fused with real-time features of different complexities and weights. This allows for dynamic adjustment of recognition features across multiple industrial camera runtimes, continuously optimizing the features and dynamically improving the recognition rate. It is suitable for industrial camera recognition tasks with high complexity conditions.

[0179] A third aspect of the present invention also provides a computer-readable storage medium comprising an industrial camera operation optimization program based on a complex scene, wherein when the industrial camera operation optimization program based on a complex scene is executed by a processor, the steps of the industrial camera operation optimization method based on a complex scene as described in any of the preceding claims are implemented.

[0180] This invention discloses a method and system for optimizing the operation of an industrial camera in complex scenes. The method includes: using an industrial camera, dividing the recognition period and acquiring video streams and environmental parameters in the first running phase; extracting keyframes to generate an image set; and forming operating feature parameters by combining the environmental parameters with feature extraction and target recognition. The operating feature parameters are then grouped using fuzzy c-means clustering, and the complexity of the recognition period is set. High-complexity time periods are selected and a complex feature set is constructed. A GAN model is used to perform data augmentation simulation on the complex feature set to generate a simulated feature library. In the second running phase, complex scene time periods are detected in real time, and dynamic fusion recognition is performed using the simulated feature library. This invention effectively improves the dynamic recognition rate of industrial cameras in complex scenes through a feature simulation-driven adaptive feature fusion mechanism, enhances the robustness and intelligence of the system, and achieves efficient operation of the industrial camera.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0182] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0183] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0184] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0186] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing the operation of industrial cameras in complex scenarios, characterized in that, include: S101: Based on the first running phase of the industrial camera, multiple recognition cycles are set, and in each recognition cycle, video stream data and environmental parameters are acquired; S102: Extract keyframes from video stream data to obtain an image set, extract image features and identify targets from the image set, set scene parameters for the image features of the target images, and generate running feature parameters based on the scene parameters and environment parameters; S103: Vectorize the running feature parameters for each recognition cycle, and group the running feature parameters and recognition cycles based on fuzzy c-means clustering. Set the complexity of the recognition cycle based on the distance between the running feature parameters and the cluster center. S104: Select high-complexity recognition periods as complex scene time periods, extract and store image features from the image set to form a complex feature set, and learn features from the complex feature set based on the GAN generative model to generate simulated features. S105: In the second runtime phase, the complex scene period is identified in real time and marked as the second scene period. The fusion weight is dynamically set according to the similarity of the runtime feature parameters between the second scene period and the complex scene period. Based on the fusion weight, the real-time recognition features and simulated features are dynamically fused and recognized to obtain the real-time recognition result. Wherein, S102, For each recognition cycle, keyframes are extracted from the video stream data to obtain an image set; The image set is preprocessed by grayscale conversion, smoothing, noise reduction, and normalization. The image recognition module identifies and labels the target object to obtain the target region. The number of objects in the target region is then identified to obtain the quantity parameters. Based on the Sobel operator, contour feature analysis and contour quantity statistics are performed on the target region to obtain contour parameters; In the target area, the color differences in the area are analyzed based on the color histogram, and the color difference degree is evaluated in the form of data variance to obtain the color difference parameter; Scene parameters are generated based on quantity parameters, contour parameters, and color difference parameters; The scene parameters and environment parameters are integrated to obtain the runtime characteristic parameters; Specifically, S104 is as follows: The complexity is compared with the preset value, and the recognition period with high complexity is selected and marked as the complex scene period. Based on all the time periods of the complex scene, keyframes for each time period are extracted from the image set to obtain the complex image set; The image recognition module identifies and extracts features of target objects from complex image sets, and extracts the contour features and color features of target objects based on the Sobel operator and color histogram, respectively. In a complex image set, a complex scene time period is analyzed, and spatial relationship features are obtained by extracting and storing the spatial relationship information between the target object and the background object. Contour features, color features, and spatial relationship features are considered as a complex feature in a recognition cycle; A complex feature set is obtained by statistically analyzing all time periods of complex scenarios.

2. The method for optimizing the operation of industrial cameras in complex scenes according to claim 1, characterized in that, Specifically, S101 is as follows: During the first run of the industrial camera, multiple recognition cycles are set based on the recognition task; In each recognition cycle, video streams of the target area are acquired and the target environmental state is identified based on industrial cameras, resulting in video stream data and environmental parameters; Environmental parameters include light intensity, object velocity, and background complexity information.

3. The method for optimizing the operation of industrial cameras in complex scenes according to claim 1, characterized in that, Specifically, S103 is as follows: For each recognition cycle, the corresponding running feature parameters are vectorized to obtain the recognition state vector; Each recognition state vector is used as a data point, and the number of data points is set to m. Based on the fuzzy c-means clustering algorithm, n cluster centers are set, an n×m membership matrix R is randomly initialized, and the distance from each data point to the cluster center is repeatedly calculated using Euclidean distance. The cluster centers and membership matrix R are updated using the shortest distance until the preset number of iterations is reached. Based on the membership matrix R, determine the maximum membership degree of the cluster center to which each data point belongs, and confirm the cluster group of each data point to obtain n cluster groups; In n clusters, the classification results based on the recognition state vector are mapped to the running feature parameters and the recognition period to obtain multiple recognition periods; For each recognition cycle, calculate the mean distance between the data point corresponding to the running feature parameter and the remaining n-1 cluster centers, and obtain the recognition cycle complexity based on the mean distance.

4. The method for optimizing the operation of industrial cameras in complex scenes according to claim 1, characterized in that, S104 includes: A generative model based on GAN is constructed, which includes a generator and a discriminator, and the mean squared error is used as the loss function. The complex feature set is imported into the generator as real data, and the complexity of different complex features is imported into the generative model as training weights. The generator then generates simulated training features. The simulated training features are imported into the discriminator for probability discrimination, and the generator and discriminator parameters are updated based on the error backpropagation algorithm. Repeatedly train the generative model adversarially until the loss function converges to the preset error or the generator and discriminator reach Nash equilibrium. Generate a predetermined number of simulated features based on the trained generative model.

5. The method for optimizing the operation of industrial cameras in complex scenarios according to claim 1, characterized in that, Specifically, S105 is: During the second runtime phase, complex scene time periods are identified in real time and marked as the second scene time periods; In the first runtime period, the most complex scenario period is marked as the first scenario period; Based on the similarity of the operational characteristic parameters between the second scene time period and the first scene time period, the fusion weight of the second scene time period is dynamically set; Based on industrial cameras, image features of all second scene time periods are acquired in real time and labeled as real-time recognition features; The real-time recognition features and simulated features are dynamically fused. The fusion process introduces a linear weighted fusion method and combines fusion weights to perform feature vectorization and feature fusion of the real-time recognition features and simulated features. Based on the fused features, the target object is identified in real time to obtain the real-time recognition result.

6. An industrial camera operation optimization system based on complex scenes, characterized in that, The system includes a memory and a processor. The memory includes an industrial camera operation optimization program based on complex scenes. When the processor executes the industrial camera operation optimization program based on complex scenes, it implements the steps of the industrial camera operation optimization method based on complex scenes as described in claim 1.

7. The industrial camera operation optimization system based on complex scenes according to claim 6, characterized in that, Specifically, S101 is as follows: During the first run of the industrial camera, multiple recognition cycles are set based on the recognition task; In each recognition cycle, video streams of the target area are acquired and the target environmental state is identified based on industrial cameras, resulting in video stream data and environmental parameters; Environmental parameters include light intensity, object velocity, and background complexity information.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes an industrial camera operation optimization program based on complex scenes. When the industrial camera operation optimization program based on complex scenes is executed by a processor, it implements the steps of the industrial camera operation optimization method based on complex scenes as described in any one of claims 1 to 5.

Images