A multi-position pan / tilt control method and device

Abstract

The present application relates to the field of pan-tilt control technology, and discloses a multi-station pan-tilt control method and device. The method includes: obtaining the object characteristic data of the target object and analyzing and processing it to obtain the multi-station pan-tilt configuration scheme and the station detection area allocation strategy; performing collaborative motion path planning to obtain the multi-station pan-tilt collaborative motion trajectory; controlling the multi-station pan-tilt to synchronously perform multi-modal imaging scanning to obtain a multi-station multi-modal image data set; performing comprehensive scene analysis on the multi-station multi-modal image data set to obtain the multi-station pan-tilt dynamic lighting control parameters; based on the multi-station pan-tilt dynamic lighting control parameters, performing cross-station data fusion on the multi-station multi-modal image data set to obtain a comprehensive digital model of the object; performing self-optimization analysis on the comprehensive digital model of the object and the multi-station detection process data to obtain the multi-station pan-tilt system optimization scheme and station maintenance warning information. The present application improves the control accuracy of the multi-station pan-tilt.

Description

Technical Field

[0001] The present application relates to the technical field of pan / tilt control, and in particular to a multi-station pan / tilt control method and device. Background Art

[0002] The multi-station pan-tilt head can achieve efficient and accurate detection and processing of target objects by precisely controlling the coordinated movement and synchronous operation of multiple workbenches. However, the traditional multi-station pan-tilt head control method still has many shortcomings when dealing with complex environments and changing working conditions, and it is difficult to meet the modern industry's requirements for high efficiency, high precision and multi-function.

[0003] Existing multi-station pan-tilt control systems often lack flexibility and adaptability when faced with different object characteristics and diverse inspection requirements. Traditional methods usually rely on preset motion trajectories and fixed lighting schemes, which are difficult to dynamically adjust and optimize, resulting in low inspection efficiency in complex scenarios. In addition, the collaborative control of multi-station pan-tilts is often limited by time-space conflicts between stations and improper path planning, making it impossible to achieve efficient collaborative movement and synchronous operation of multiple stations, affecting the overall system performance. Summary of the invention

[0004] The present application provides a multi-station pan-tilt control method and device for improving the control accuracy of the multi-station pan-tilt.

[0005] In a first aspect, the present application provides a multi-station pan-tilt control method, the multi-station pan-tilt control method comprising:

[0006] Obtain the object characteristic data of the target object, analyze and process it, and obtain the multi-station pan / tilt configuration plan and station detection area allocation strategy;

[0007] Based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy, collaborative motion path planning is performed to obtain a multi-station pan-tilt collaborative motion trajectory;

[0008] According to the collaborative motion trajectory of the multi-station pan-tilt platform, the multi-station pan-tilt platform is controlled to synchronously perform multi-modal imaging scanning on the target object to obtain a multi-station multi-modal image data set;

[0009] Performing comprehensive scene analysis on the multi-station multimodal image data set to obtain dynamic lighting control parameters of the multi-station pan / tilt platform;

[0010] Based on the multi-station pan-tilt dynamic lighting control parameters, cross-station data fusion is performed on the multi-station multimodal image data set to obtain a comprehensive digital model of the object;

[0011] A self-optimizing analysis is performed on the comprehensive digital model of the object and the multi-station detection process data to obtain a multi-station pan / tilt system optimization plan and station maintenance early warning information.

[0012] In a second aspect, the present application provides a multi-station pan-tilt control device, the multi-station pan-tilt control device comprising:

[0013] The acquisition module is used to acquire the object characteristic data of the target object and analyze and process it to obtain the multi-station pan-tilt configuration plan and the station detection area allocation strategy;

[0014] A planning module, used to plan a collaborative motion path based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy to obtain a multi-station pan-tilt collaborative motion trajectory;

[0015] A scanning module, used for controlling the multi-station pan-tilt table to synchronously perform multi-modal imaging scanning on the target object according to the coordinated motion trajectory of the multi-station pan-tilt table, so as to obtain a multi-station multi-modal image data set;

[0016] An analysis module, used to perform comprehensive scene analysis on the multi-station multi-modal image data set to obtain dynamic lighting control parameters of the multi-station pan / tilt platform;

[0017] A fusion module, used for performing cross-station data fusion on the multi-station multimodal image data set based on the multi-station pan-tilt dynamic lighting control parameters to obtain a comprehensive digital model of the object;

[0018] The optimization module is used to perform self-optimization analysis on the comprehensive digital model of the object and the multi-station detection process data to obtain the multi-station pan-tilt system optimization plan and station maintenance early warning information.

[0019] The third aspect of the present application provides a multi-station pan-tilt control device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the multi-station pan-tilt control device executes the above-mentioned multi-station pan-tilt control method.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned multi-station pan-tilt control method.

[0021] In the technical solution provided by the present application, by acquiring and analyzing the object characteristic data of the target object, the surface of the object can be feature analyzed and adaptively segmented, and the detection sub-area can be effectively allocated. The detection area is optimized and allocated by using multi-target clustering analysis and load balancing strategy, which improves the detection efficiency of the workstation and the operating efficiency of the overall system. Based on the multi-station pan-tilt configuration scheme and the detection area allocation strategy, collaborative motion path planning is carried out to obtain an efficient collaborative motion trajectory. Through multi-station spatiotemporal conflict detection and dynamic priority allocation, efficient collaborative motion between multiple stations is achieved, reducing interference and conflict between stations. The multi-station pan-tilt is controlled to perform multimodal imaging scanning synchronously to obtain a high-precision multimodal image data set, thereby improving the imaging quality. Adaptive dynamic focus adjustment and real-time correction and denoising processing are used to obtain a high-definition image group to ensure the accuracy and integrity of the image data. Based on comprehensive scene analysis, the lighting control parameters of the pan-tilt are dynamically adjusted, the lighting layout and illumination distribution are optimized, and the imaging quality is improved. Through dynamic ray tracing simulation and illumination error compensation, precise lighting control is achieved, reducing the impact of uneven illumination on imaging. Perform illumination correction and feature matching on multi-station multimodal image data to achieve cross-station data fusion and build a comprehensive digital model of the object. Use multi-scale feature extraction and geometric transformation to ensure the accuracy of data fusion and the integrity of model construction, and provide a high-precision digital representation of the object. Based on multi-station detection process data and object digital models, perform self-optimization analysis, propose system optimization solutions, and improve the operating efficiency of the system. Generate maintenance warning information through multi-level station status prediction and dynamic threshold analysis to prevent potential failures and ensure the reliability and stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 This is a schematic diagram of an embodiment of a multi-station pan-tilt control method in an embodiment of the present application;

[0024] Figure 2 This is a schematic diagram of an embodiment of a multi-station pan-tilt control device in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application provide a multi-station pan-tilt control method and device. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, article, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, articles, or devices.

[0026] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the multi-station pan-tilt control method in the embodiment of the present application includes:

[0027] Step S101, obtaining the object characteristic data of the target object and analyzing and processing it to obtain a multi-station pan / tilt configuration scheme and a station detection area allocation strategy;

[0028] It is understandable that the execution subject of the present application may be a multi-station PTZ control device, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0029] Specifically, the object characteristic data of the target object is obtained through sensing technology and data acquisition equipment. These data include object characteristic data such as size, shape, surface texture, color, reflectivity, etc. According to the object characteristic data, the surface of the target object is subjected to feature analysis and adaptive region segmentation. The surface of the target object is divided into multiple detection sub-regions using image processing algorithms and machine learning models to ensure that each sub-region can be detected independently and improve detection accuracy and efficiency. Multi-dimensional feature extraction is performed on each detection sub-region, including shape features, texture features, color features, geometric features, etc., to obtain a sub-region feature vector set. By analyzing the feature vectors, the detection difficulty of each detection sub-region is evaluated and prioritized. According to the similarities and differences between different feature vectors, the regional detection complexity index of each detection sub-region is calculated. This index reflects the complexity and priority of each detection sub-region in the actual detection process. Based on the regional detection complexity index, multi-target clustering analysis is performed. Multiple detection sub-regions are classified and grouped according to their complexity and characteristics to obtain an initial workstation allocation plan. Sub-regions with similar complexity are clustered together using an algorithm to optimize the overall detection efficiency. The initial station allocation plan is calculated for station load balancing and detection efficiency optimization. Station load balancing is to ensure that each station can work evenly during the detection process to avoid overloading or idleness of some stations. The detection efficiency optimization calculation is to maximize the overall detection efficiency when allocating the detection area, ensure that the detection speed and accuracy of each station are in the best state, and form a station detection area allocation strategy. According to the station detection area allocation strategy, the detection requirements and constraints of each station pan-tilt are analyzed. Detection requirements analysis refers to determining the specific detection tasks that need to be completed at each station, while constraint analysis refers to considering the limitations and requirements of each station in terms of space, time, resources, etc. In this process, the station function requirement specifications of each station are generated to clarify the functions and performance indicators that each station needs to have. Based on the station function requirement specifications, the pan-tilt hardware configuration and parameter optimization are performed on each station pan-tilt. The pan-tilt hardware configuration includes selecting appropriate sensors, cameras, lighting equipment, mobile mechanisms, etc., and reasonably installing and debugging these hardware. Parameter optimization refers to adjusting and optimizing the motion parameters, imaging parameters, and lighting parameters of the gimbal according to the detection requirements and environmental conditions to ensure that the gimbal can work stably and accurately during the detection process. The multi-station gimbal configuration plan and the station detection area allocation strategy are obtained.

[0030] Step S102, performing collaborative motion path planning based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy to obtain the multi-station pan-tilt collaborative motion trajectory;

[0031] Specifically, according to the multi-station pan-tilt configuration scheme, kinematic modeling is performed for each station pan-tilt. By establishing the kinematic model of the station pan-tilt, the motion characteristics of each station pan-tilt in space are described, including its key parameters such as joint parameters, motion range, speed and acceleration. Through these models, the motion behavior of the station pan-tilt is accurately simulated and predicted. Based on the station detection area allocation strategy, adaptive coverage path planning is performed. By analyzing the specific distribution and characteristics of each detection area, an initial detection path set is generated. These paths are designed to cover all detection areas and ensure the comprehensiveness and accuracy of the detection task. Multi-station spatiotemporal conflict detection and analysis are performed on the initial detection path set to identify potential conflict point sets. The potential conflict point set represents the area that may cause interference or collision of the multi-station pan-tilt motion. According to the potential conflict point set and the detection area priority, the initial detection path set is dynamically prioritized and time-series adjusted to generate an initial collaborative path plan, which avoids conflicts as much as possible in time and space and gives priority to important detection areas. The initial collaborative path plan is calculated by multi-station parallel kinematic inverse solution to obtain the joint motion sequence. Based on the joint motion sequence, a smooth motion trajectory is generated through polynomial interpolation and spline curve fitting to ensure the continuity and stability of the motion of the workstation gimbal during operation and avoid the impact of sudden speed and acceleration changes on the system. The smooth motion trajectory is analyzed for the spatiotemporal coordination between multiple workstations, and the coordination index and coordination efficiency score between each workstation are calculated to reflect the overall coordination and efficiency of the multi-station gimbal when performing inspection tasks. Based on these data, global iterative optimization and adjustment are performed to improve the coordination and efficiency of the system and obtain a multi-station highly coordinated motion trajectory.

[0032] Step S103, according to the collaborative motion trajectory of the multi-station pan-tilt platform, controlling the multi-station pan-tilt platform to synchronously perform multi-modal imaging scanning on the target object to obtain a multi-station multi-modal image data set;

[0033] Specifically, the time series decomposition and interpolation of the collaborative motion trajectory of the multi-station pan-tilt is performed. Through the time series decomposition method, the motion trajectory of each station pan-tilt is refined to obtain the high-precision discrete time point position sequence of each station pan-tilt. According to the high-precision discrete time point position sequence and the station detection area allocation strategy, the multimodal imaging device of each station pan-tilt is optimized and calculated. The optimal imaging parameter set is determined by analyzing the specific position and detection requirements of each pan-tilt in each detection area. These parameters include exposure time, light source brightness, lens focal length, sensor sensitivity, etc., to ensure that each imaging device can work in the best state in different detection areas and obtain high-quality image data. Based on the optimal imaging parameter set and the multi-station pan-tilt configuration scheme, the cross-station synchronous trigger timing design is performed for the multimodal imaging device of each station pan-tilt. Through the globally optimized timing control, the multimodal imaging synchronization control instruction is generated to ensure that the imaging devices of each station pan-tilt can work synchronously at the same time, avoiding data deviation and error caused by asynchronous timing. The globally optimized multimodal imaging synchronization control instruction is parallel processed and task allocated to obtain a collaborative parallel control instruction stream. These instruction streams are transmitted and processed in real time in the system to ensure that each station gimbal can accurately perform the predetermined imaging task. According to the collaborative parallel control instruction stream and the kinematic model of the station gimbal, the drive system of each station gimbal is calculated for real-time motion error compensation to obtain the target real-time position control signal. The real-time motion error compensation calculation can effectively correct the position deviation of the gimbal during the movement process to ensure the accuracy and stability of the imaging process. Based on the target real-time position control signal, the imaging system of each station gimbal is adaptively adjusted for dynamic focus. Through dynamic focus adjustment, multi-scale focal plane compensation parameters are obtained to ensure that the imaging system can obtain clear images at different distances and angles. At the same time, according to the multi-scale focal plane compensation parameters and the station coordination index, the acquired original multimodal image data is corrected and denoised in real time to obtain a high-definition image group. Multi-scale and multi-directional feature extraction is performed on the high-definition image group to extract representative features and form a feature vector set. Based on the multi-dimensional feature vector set and the station detection area allocation strategy, cross-station area matching and spatial registration are performed to obtain a multi-station spatial alignment image set. Time series association analysis and data completion are performed on the multi-station spatially aligned image set to obtain a multi-station multimodal image dataset. Through time series association analysis, image data at different time points are identified and associated, and data missing due to synchronization problems are completed to form a complete and coherent multi-station multimodal image dataset.

[0034] Step S104, performing comprehensive scene analysis on the multi-station multi-modal image data set to obtain multi-station pan / tilt dynamic lighting control parameters;

[0035] Specifically, modality separation and multi-scale decomposition are performed on the multi-station multimodal image data set. Through the image processing algorithm, the images of different modes are separated and decomposed into images of multiple scales to form a multi-level image pyramid structure. Adaptive local contrast analysis is performed according to the multi-level image pyramid structure. By analyzing the local contrast changes of each level of the image, the surface reflection characteristic map of the target object of multimodal fusion is obtained. The reflection characteristic map can describe the optical properties of the surface of the target object. Based on the surface reflection characteristic map of the target object of multimodal fusion, a multi-dimensional lighting demand assessment is performed. By evaluating the reflection characteristics of the surface of the target object under different lighting conditions, a regional lighting parameter demand matrix is ​​generated. The matrix reflects the light intensity, angle and spectral characteristics required by different regions during the imaging process. The regional lighting parameter demand matrix is ​​analyzed, and the regions with similar requirements are divided into composite regional groups to simplify the subsequent lighting control and optimization process. According to the composite regional group, the light source type and position of the multi-spectral lighting equipment of each station pan / tilt are globally optimized. By comprehensively considering the lighting requirements and equipment configuration of each station, the first multi-station collaborative lighting layout plan is generated. The scheme determines the type, quantity and position arrangement of light sources for each station to ensure that the lighting conditions of each station meet the imaging requirements. Based on the first multi-station collaborative lighting layout scheme and the multi-station pan-tilt collaborative motion trajectory, a dynamic ray tracing simulation is performed. By simulating the propagation path of light under different time and space conditions, a time-varying theoretical illumination distribution atlas is generated to predict the actual illumination distribution of each station pan-tilt under different motion trajectories and lighting conditions. The time-varying theoretical illumination distribution atlas is compared and analyzed with the multi-station multimodal image dataset in time and space to identify the difference between the actual illumination distribution and the theoretical distribution, and to generate a dynamic illumination error compensation coefficient matrix. This matrix is ​​used to correct the deviation in actual illumination and ensure imaging quality. According to the dynamic illumination error compensation coefficient matrix, the first multi-station collaborative lighting layout scheme is iteratively optimized across stations. By continuously adjusting and optimizing the type, position and illumination parameters of the light source, a second multi-station collaborative lighting layout scheme is generated. Based on the time series information of the second multi-station collaborative lighting layout scheme, the multi-station pan-tilt collaborative motion trajectory and the multi-station multimodal image dataset, dynamic dimming sequence and stroboscopic synchronization calculation are performed. The generated dynamic dimming sequence and strobe synchronization instructions ensure that the brightness and strobe of the light sources at each station can change synchronously during the imaging process to adapt to different imaging requirements. The multi-modal collaborative lighting control instruction set is obtained. The multi-modal collaborative lighting control instruction set is cross-station energy consumption balanced and the instruction flow is optimized to ensure that the energy consumption of each station is balanced during the lighting process, the instruction flow is smooth and efficient, and the dynamic lighting control parameters of the multi-station pan-tilt are obtained.

[0036] Step S105, based on the multi-station pan-tilt dynamic lighting control parameters, cross-station data fusion is performed on the multi-station multimodal image data set to obtain a comprehensive digital model of the object;

[0037] Specifically, according to the dynamic lighting control parameters of the multi-station pan-tilt, the multi-station multimodal image data set is corrected for illumination non-uniformity. Through the illumination correction algorithm, the image brightness and contrast inconsistency caused by illumination changes in each station during the imaging process is eliminated to obtain a corrected image set. Multi-scale and multi-directional feature extraction and cross-station matching are performed on the corrected image set. Through the image processing algorithm, image features are extracted at different scales and directions to generate a feature correspondence matrix. The matrix records the correspondence between feature points of each image. Based on the feature correspondence matrix, geometric transformation and precise registration are performed to accurately align the images of different stations in space to form a unified multi-station aligned image set. The multi-station aligned image set is subjected to multimodal data normalization processing. Through data standardization technology, image data of different modalities are converted into a unified standardized multimodal data set. The standardization process can eliminate the differences between different modalities, so that data from different sources can be processed and analyzed on the same platform. According to the standardized multimodal data set, a high-dimensional feature tensor of items that integrates station information is constructed. This feature tensor integrates the detection data of different workstations and modalities to form a high-dimensional feature representation. The high-dimensional feature tensor of the item that integrates the workstation information is subjected to cross-workstation tensor decomposition and feature importance evaluation. Through tensor decomposition technology, the high-dimensional feature tensor is decomposed into multiple low-dimensional sub-tensors to extract the core feature representation of the item. Based on the core feature representation of the item, an initial digital model of the item containing the workstation detection information is constructed. The initial model integrates the detection data and core features of each workstation to form a preliminary digital representation. The initial digital model of the item is subjected to multi-workstation semantic segmentation and attribute comprehensive mapping. Through semantic segmentation technology, the digital model is divided into different functional areas, and the attributes of each area are comprehensively mapped to form a complete item attribute map. By integrating the detection data and attribute information of different workstations, a comprehensive digital model of the item is obtained.

[0038] Step S106: self-optimize and analyze the comprehensive digital model of the object and the multi-station detection process data to obtain a multi-station pan / tilt system optimization solution and station maintenance warning information.

[0039] Specifically, the comprehensive digital model of the object is subjected to multi-scale feature decomposition and cross-station statistical analysis. By performing multi-level feature decomposition on the model, the key features of the object are extracted, and statistical analysis is performed in the detection data of each station to obtain the distribution map of the key features of the object. A comprehensive evaluation is performed based on the distribution map of the key features of the object to generate a multi-dimensional station detection efficiency matrix. This matrix can reflect the efficiency and performance of each station in different detection tasks. A multivariate correlation analysis is performed on the multi-dimensional station detection efficiency matrix to construct a station motion-detection efficiency dynamic relationship model. Through this model, the dynamic relationship between the motion state of the station and its detection efficiency is analyzed. Based on this model, parameter testing is performed to identify the performance bottleneck in the system and obtain the performance bottleneck identification result. The performance bottleneck identification result can reveal the key problems and limitations of the system during operation and provide direction for system optimization. According to the performance bottleneck identification result, an adaptive parameter sensitivity analysis is performed on the multi-station pan-tilt configuration scheme to evaluate the impact of each parameter on the system performance and obtain suggestions for system optimization direction. These suggestions are evaluated and sorted across stations, and a priority ranking is formed by comprehensively considering the actual situation of each station to obtain a phased system optimization execution plan. Based on the phased system optimization execution plan, a multi-level station status prediction model is constructed. Through this model, the state changes of each station in future operation are predicted, and a multi-dimensional performance degradation trend chart is generated. This trend chart shows the trend of system performance changes over time, which can help identify potential performance degradation risks. Dynamic threshold analysis and cumulative risk assessment are performed on the multi-dimensional performance degradation trend chart to determine the maintenance requirements of each station and generate station-level maintenance warning indicators. Based on these indicators, time-varying reliability analysis is performed to obtain a system maintenance warning strategy. This strategy can identify and prevent potential failures in the system in advance and ensure the continuous and stable operation of the system. The system maintenance warning strategy and the phased system optimization execution plan are integrated to form a comprehensive multi-station PTZ system optimization solution and station maintenance warning information.

[0040] In the embodiment of the present application, by acquiring and analyzing the object characteristic data of the target object, the surface of the object can be feature analyzed and adaptively segmented, and the detection sub-area can be effectively allocated. The detection area is optimized and allocated by using multi-target clustering analysis and load balancing strategy, which improves the detection efficiency of the workstation and the operating efficiency of the overall system. Based on the multi-station pan-tilt configuration scheme and the detection area allocation strategy, collaborative motion path planning is performed to obtain an efficient collaborative motion trajectory. Through multi-station spatiotemporal conflict detection and dynamic priority allocation, efficient collaborative motion between multiple stations is achieved, reducing interference and conflict between stations. The multi-station pan-tilt is controlled to perform multimodal imaging scanning synchronously to obtain a high-precision multimodal image data set, thereby improving the imaging quality. Adaptive dynamic focus adjustment and real-time correction and denoising processing are used to obtain a high-definition image group to ensure the accuracy and integrity of the image data. Based on comprehensive scene analysis, the lighting control parameters of the pan-tilt are dynamically adjusted, the lighting layout and illumination distribution are optimized, and the imaging quality is improved. Through dynamic ray tracing simulation and illumination error compensation, precise lighting control is achieved, reducing the impact of uneven illumination on imaging. Perform illumination correction and feature matching on multi-station multimodal image data to achieve cross-station data fusion and build a comprehensive digital model of the object. Use multi-scale feature extraction and geometric transformation to ensure the accuracy of data fusion and the integrity of model construction, and provide a high-precision digital representation of the object. Based on multi-station detection process data and object digital models, perform self-optimization analysis, propose system optimization solutions, and improve the operating efficiency of the system. Generate maintenance warning information through multi-level station status prediction and dynamic threshold analysis to prevent potential failures and ensure the reliability and stability of the system.

[0041] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0042] (1) Obtaining the object characteristic data of the target object, and performing feature analysis and adaptive region segmentation on the surface of the target object based on the object characteristic data to obtain multiple detection sub-regions;

[0043] (2) Perform multi-dimensional feature extraction on each detection sub-region to obtain a sub-region feature vector set. Based on the sub-region feature vector set, perform detection difficulty evaluation and priority sorting on each detection sub-region to obtain a regional detection complexity index.

[0044] (3) Based on the regional detection complexity index, a multi-objective clustering analysis is performed on multiple detection sub-areas to obtain the initial workstation allocation plan. The workstation load balancing and detection efficiency optimization calculations are performed on the initial workstation allocation plan to obtain the workstation detection area allocation strategy;

[0045] (4) According to the workstation detection area allocation strategy, the detection requirements and constraints of each workstation pan-tilt are analyzed to obtain the workstation function requirement specifications. Based on the workstation function requirement specifications, the pan-tilt hardware configuration and parameter optimization of each workstation pan-tilt are performed to obtain a multi-station pan-tilt configuration plan.

[0046] Specifically, the object characteristic data of the target object is obtained through a high-resolution camera and an optical sensor. This data includes the object's size, shape, color, texture, and surface reflectivity. Assuming that the surface reflectivity of the object varies significantly in different areas, the reflectivity is used. To indicate the surface of an object at point The image data is preprocessed to remove noise and interference to ensure the accuracy and reliability of the data. Image processing technology is used to analyze the surface of the self-labeled object, and the surface is segmented into multiple detection sub-regions using the adaptive region segmentation algorithm. Feature analysis includes edge detection, texture analysis, and color segmentation. These techniques are used to identify and extract key features of the object surface. Adaptive region segmentation can use the K-means clustering algorithm to divide the pixels in the image into sub-regions. Set the initial number of clusters to , the algorithm divides the pixels in the image into Through continuous iteration, the distance between each pixel and the cluster center is calculated, and the pixel is redistributed according to the minimum distance principle. Finally, detection sub-regions. The K-means clustering algorithm formula is:

[0047] ;

[0048] in, It is Clusters, It is For each detection sub-region, multi-dimensional feature extraction is performed. Assume that the feature vector extracted from the image is ,in Indicates The sub-region These features can include color histogram, texture descriptor, edge strength, etc. By extracting the features of each detected sub-region, we can get the sub-region feature vector set ,in Indicates the number of sub-regions. According to the sub-region feature vector set, the detection difficulty of each detection sub-region is evaluated and prioritized. The detection difficulty can be evaluated by factors such as the complexity of the feature vector and the texture change in the region. Assume that the detection difficulty index is the eigenvector Function:

[0049] ;

[0050] in, It is a feature By analyzing the feature vector of each sub-region, the detection difficulty index is calculated, and the priority is sorted according to the detection difficulty index to obtain the regional detection complexity index. ,in Indicates The detection complexity of each sub-region is calculated. Based on the regional detection complexity index, a multi-objective clustering analysis is performed on multiple detection sub-regions to obtain an initial workstation allocation plan. Assuming that a hierarchical clustering algorithm is used for multi-objective clustering analysis, sub-regions with similar detection complexity are divided into the same cluster by constructing a clustering tree. The clustering process can be performed by calculating the similarity matrix between sub-regions. ,in Indicates sub-regions and The similarity between the sub-areas is calculated. The initial workstation allocation scheme is obtained by performing hierarchical clustering on the similarity matrix. The formula is:

[0051] ;

[0052] in, It is Clusters, It is The initial workstation allocation plan is used to perform workstation load balancing and detection efficiency optimization calculations. The purpose of load balancing is to ensure that the detection tasks of each workstation are balanced and to avoid overloading or idleness of some workstations. Load balancing and detection efficiency optimization calculations can be performed through optimization algorithms, such as linear programming. Assume that the detection tasks of the workstation are ,in Indicates The inspection task of each workstation is adjusted through the optimization algorithm to obtain the workstation inspection area allocation strategy. These include space, time, and resource constraints:

[0053] ;

[0054] in, is the spatial constraint, It is a time constraint. is a resource constraint. The constraints are:

[0055] ;

[0056] in, It's a workstation No. The detection complexity of the task is It's a workstation capacity. According to the workstation detection area allocation strategy, the detection requirements and constraints of each workstation gimbal are analyzed to obtain the workstation function requirement specifications. Based on these specifications, the gimbal hardware configuration and parameter optimization are performed on each workstation gimbal to ensure that the gimbal can work stably and accurately during the detection process. The gimbal hardware configuration includes selecting appropriate sensors, cameras, lighting equipment, etc., and reasonably installing and debugging these hardware. Parameter optimization refers to adjusting and optimizing the motion parameters, imaging parameters, lighting parameters, etc. of the gimbal according to the detection requirements and environmental conditions to ensure that the gimbal can work stably and accurately during the detection process.

[0057] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0058] (1) According to the multi-station pan-tilt configuration scheme, kinematic modeling is performed on each station pan-tilt to obtain the kinematic model of the station pan-tilt;

[0059] (2) Based on the workstation detection area allocation strategy, adaptive coverage path planning is performed to obtain the initial detection path set, and multi-station spatiotemporal conflict detection and analysis are performed on the initial detection path set to obtain the potential conflict point set;

[0060] (3) According to the potential conflict point set and the detection area priority, the initial detection path set is dynamically prioritized and the timing is adjusted to obtain the initial collaborative path solution;

[0061] (4) Perform multi-station parallel kinematic inverse solution calculation on the initial collaborative path plan to obtain the joint motion sequence, and perform polynomial interpolation and spline curve fitting based on the joint motion sequence to obtain a smooth motion trajectory;

[0062] (5) The spatiotemporal coordination of multiple workstations is analyzed on the smooth motion trajectory to obtain the workstation coordination index and coordination efficiency score. Based on the workstation coordination index and coordination efficiency score, global iterative optimization and adjustment are performed to obtain a multi-workstation highly coordinated motion trajectory.

[0063] Specifically, according to the multi-station gimbal configuration scheme, a kinematic model is built for each station gimbal to describe the motion characteristics of the gimbal in space, including position, velocity, acceleration, etc. Set the joint variables of the gimbal to ,in Indicates The relationship between the joint variables and the position and orientation of the end effector is described using the forward kinematics equation:

[0064] ;

[0065] in, is the transformation matrix of the end effector, which contains the position and orientation information. The inverse kinematics equation is used to calculate the joint variables when the end effector position and orientation are given:

[0066] ;

[0067] Adaptive coverage path planning is performed based on the workstation detection area allocation strategy. The goal of adaptive coverage path planning is to ensure that each detection area can be effectively covered by the PTZ and generate an initial detection path set. Assume that the detection area is , each detection path Corresponding to a detection area By optimizing the algorithm, the starting point and end point of each path are calculated to ensure that the path covers the entire detection area:

[0068] ;

[0069] in, Represents the detection point on the path. Perform multi-station spatiotemporal conflict detection and analysis on the initial detection path set to identify the potential conflict point set. Potential conflict points refer to the locations where different station pan / tilts may have physical conflicts at the same time. By performing spatiotemporal analysis on the detection path, calculate the timestamp of each detection point , and detect whether the spatial positions of different paths overlap in the same time period:

[0070] ;

[0071] in, Represents the potential conflict point set. According to the potential conflict point set and the detection area priority, the initial detection path set is dynamically prioritized and the timing is adjusted to generate the initial collaborative path solution. Priority allocation is based on the importance of each detection area, and the execution order and time interval of the path are adjusted to avoid potential conflicts. Suppose the priority of the detection area is , by adjusting the time parameters of the path , ensuring that high priority areas are inspected first while avoiding conflict points:

[0072] ;

[0073] in, Represents the time adjustment. Perform multi-station parallel kinematic inverse calculation on the initial collaborative path plan to obtain the joint motion sequence. Calculate the joint variables at each time point through the inverse kinematics equation , and get the joint motion sequence:

[0074] ;

[0075] Based on the joint motion sequence, polynomial interpolation and spline curve fitting are performed to generate a smooth motion trajectory. Assuming that cubic spline curve is used for fitting, the joint variables A function that can be expressed as a cubic polynomial:

[0076] ;

[0077] Determine the polynomial coefficients by fitting the joint motion sequence , and a smooth motion trajectory is obtained. The temporal and spatial coordination analysis among multiple workstations is performed on the smooth motion trajectory, and the workstation coordination index and coordination efficiency score are calculated. The coordination index reflects the coordination degree of multiple workstation pan / tilts when performing the inspection task, and the coordination efficiency score measures the overall efficiency of the inspection process. Suppose the workstation coordination index is , the collaborative efficiency score is By analyzing the smooth motion trajectory, calculating these indicators, and performing global iterative optimization and adjustment based on them, we can finally obtain a multi-station high-coordination motion trajectory.

[0078] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0079] (1) Perform time series decomposition and interpolation on the collaborative motion trajectory of the multi-station pan-tilt system to obtain a high-precision discrete time point position sequence of each pan-tilt system;

[0080] (2) Based on the high-precision discrete time point position sequence and the workstation detection area allocation strategy, the multimodal imaging equipment of each workstation pan / tilt is optimized and calculated to obtain the optimal imaging parameter set;

[0081] (3) Based on the optimal imaging parameter set and the multi-station pan-tilt configuration scheme, the cross-station synchronous trigger timing design of the multimodal imaging equipment of each station pan-tilt is performed to obtain the globally optimized multimodal imaging synchronous control instructions;

[0082] (4) Parallel processing and task allocation are performed on the globally optimized multi-modal imaging synchronization control instructions to obtain a coordinated parallel control instruction stream. Based on the coordinated parallel control instruction stream and the kinematic model of the workstation pan-tilt platform, real-time motion error compensation calculation is performed on the drive system of each workstation pan-tilt platform to obtain the target real-time position control signal;

[0083] (5) Based on the target real-time position control signal, the imaging system of each workstation pan / tilt is adaptively and dynamically focused to obtain multi-scale focal plane compensation parameters. Based on the multi-scale focal plane compensation parameters and the workstation coordination index, the acquired original multimodal image data is corrected and denoised in real time to obtain a high-definition image group.

[0084] (6) Perform multi-scale and multi-directional feature extraction on the high-definition image group to obtain a multi-dimensional feature vector set, and perform cross-station area matching and spatial registration based on the multi-dimensional feature vector set and the station detection area allocation strategy to obtain a multi-station spatially aligned image set;

[0085] (7) Perform time series correlation analysis and data completion on the multi-station spatially aligned image set to obtain a multi-station multimodal image dataset.

[0086] Specifically, the multi-station pan-tilt collaborative motion trajectory is decomposed and interpolated in time series. The continuous motion trajectory is decomposed into a discrete time point sequence to facilitate high-precision motion control and data acquisition. Assume that the motion trajectory of the pan-tilt is , through time series decomposition, we can get the discrete time point position sequence The polynomial interpolation method is used to interpolate discrete points and generate a high-precision time point position sequence. The polynomial interpolation formula is:

[0087] ;

[0088] in, It's time The trajectory position at the time, are the coefficients of the interpolation polynomial, is the order of the interpolation polynomial. According to the high-precision discrete time point position sequence and the station detection area allocation strategy, the multimodal imaging equipment of each station pan / tilt is optimized and calculated. The optimization of imaging parameters includes exposure time, light source intensity, focal length, etc. Assume that the imaging parameter set is , through the optimization algorithm, the optimal parameters of each imaging device at different time points are calculated. The optimization goal can be to maximize the clarity and contrast of the image, the formula is as follows:

[0089] ;

[0090] in, is the image quality function, is the weight coefficient, It is The contribution of each imaging parameter to the image quality. Based on the optimal imaging parameter set and the multi-station pan-tilt configuration scheme, the multi-modal imaging equipment of each station pan-tilt is designed across stations to obtain the globally optimized multi-modal imaging synchronization control instructions. The purpose of the synchronization trigger timing design is to ensure that the imaging equipment of each station works synchronously at the same time point to avoid image errors caused by timing asynchrony. The synchronization control instruction can be expressed as ,in Indicates The triggering moment of each imaging device. The globally optimized multi-modal imaging synchronization control instructions are processed in parallel and task assigned to obtain a collaborative parallel control instruction flow. The purpose of parallel processing is to improve the execution efficiency of the system, and task assignment ensures that the imaging devices of each station pan / tilt can work according to the optimized timing. The collaborative parallel control instruction flow can be expressed as ,in Indicates According to the coordinated parallel control instruction flow and the kinematic model of the workstation pan-tilt, the real-time motion error compensation calculation is performed on the drive system of each workstation pan-tilt to obtain the target real-time position control signal. The purpose of error compensation is to correct the deviation caused by mechanical error or environmental change during the movement of the pan-tilt. Assume that the target position is , the actual location is , the error is , through error compensation calculation, adjust the control signal, the formula is as follows:

[0091] ;

[0092] in, is the compensated control signal, is the compensation coefficient. Based on the target real-time position control signal, the imaging system of each station pan / tilt is adaptively adjusted to obtain multi-scale focal plane compensation parameters to ensure that clear images can be obtained at different distances and angles. The focal plane compensation parameters can be expressed as ,in Indicates The focal length adjustment amount at each time point. According to the multi-scale focal plane compensation parameters and the station coordination index, the acquired original multimodal image data is corrected and denoised in real time to obtain a high-definition image group. Multi-scale and multi-directional feature extraction is performed on the high-definition image group to obtain a multi-dimensional feature vector set. Meaningful features are identified from the image for subsequent analysis and processing. Assume that the feature vector is ,in Indicates The feature vector of each image is obtained. Based on the multi-dimensional feature vector set and the workstation detection area allocation strategy, cross-workstation area matching and spatial registration are performed to obtain a multi-workstation spatial alignment image set. The purpose of matching and registration is to align the image data of different workstations to form a complete spatial image. The displacement and rotation parameters of each image are calculated through the matching algorithm. The formula is as follows:

[0093] ;

[0094] in, is the aligned image, is the rotation matrix, is the translation vector. Time series association analysis and data completion are performed on the multi-station spatially aligned image set to obtain a multi-station multimodal image dataset. The purpose of time series association analysis is to identify the relationship between images at different time points, and data completion is used to fill in the blank areas caused by missing data to generate a complete multimodal image dataset.

[0095] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0096] (1) Perform modal separation and multi-scale decomposition on the multi-station multi-modal image dataset to obtain a multi-level image pyramid structure. Adaptive local contrast analysis is performed based on the multi-level image pyramid structure to obtain a multi-modal fusion target object surface reflection characteristic map.

[0097] (2) Based on the multi-modal fusion of the target object surface reflectance characteristic map, a multi-dimensional lighting demand assessment is performed to obtain the regional lighting parameter demand matrix, and the regional lighting parameter demand matrix is ​​analyzed to obtain a composite regional group;

[0098] (3) According to the composite area group, the light source type and position of the multi-spectral lighting equipment of each workstation pan / tilt are globally optimized and allocated to obtain the first multi-station collaborative lighting layout plan;

[0099] (4) Based on the first multi-station collaborative lighting layout scheme and the multi-station pan-tilt collaborative motion trajectory, a dynamic ray tracing simulation is performed to obtain a time-varying theoretical illumination distribution atlas. A spatiotemporal comparative analysis is performed between the time-varying theoretical illumination distribution atlas and the multi-station multimodal image dataset to obtain a dynamic illumination error compensation coefficient matrix.

[0100] (5) According to the dynamic illumination error compensation coefficient matrix, the first multi-station collaborative lighting layout scheme is iteratively optimized across stations to obtain a second multi-station collaborative lighting layout scheme;

[0101] (6) Based on the second multi-station collaborative lighting layout scheme, the multi-station pan-tilt collaborative motion trajectory, and the time series information of the multi-station multimodal image data set, dynamic dimming sequence and stroboscopic synchronization calculation are performed to obtain the multimodal collaborative lighting control instruction set;

[0102] (7) The multi-modal collaborative lighting control instruction set is used to balance energy consumption across workstations and optimize the instruction flow to obtain the dynamic lighting control parameters of the multi-station pan-tilt system.

[0103] Specifically, modality separation and multi-scale decomposition are performed on the multi-station multi-modal image dataset. Through modality separation, the image data of different imaging modalities are separated and the independent information of each modality is extracted. Multi-scale decomposition is performed to decompose the image into representations of different scales to obtain a multi-level image pyramid structure. Suppose the image Represents the surface image of the target object, and obtains a pyramid structure through multi-scale decomposition ,in Indicates Based on the multi-level image pyramid structure, adaptive local contrast analysis is performed to calculate the local contrast of each image block and generate a contrast-enhanced image. It can be calculated by the following formula:

[0104] ;

[0105] in, is the average value of the image patch, is the standard deviation of the image block. By performing contrast analysis and enhancement on each image block, a multi-modal fusion target object surface reflection characteristic map is obtained. Based on the multi-modal fusion target object surface reflection characteristic map, a multi-dimensional lighting demand assessment is performed to generate a regional lighting parameter demand matrix. Lighting parameter demand matrix Represents the lighting requirements of each area, including light intensity, spectrum, and illumination angle. By analyzing the lighting requirements of each area, composite area groups are obtained, which represent areas with similar lighting requirements. According to the composite area groups, the multi-spectral lighting equipment of each station pan / tilt is globally optimized for light source type and position, and the first multi-station collaborative lighting layout plan is generated. The optimization goal is to maximize lighting uniformity and lighting efficiency, and the formula is as follows:

[0106] ;

[0107] in, is the illumination uniformity function, is the weight coefficient, It is The illumination uniformity of each area is measured. Based on the first multi-station collaborative lighting layout scheme and the multi-station pan-tilt collaborative motion trajectory, dynamic ray tracing simulation is performed to generate a time-varying theoretical illumination distribution atlas. Through the ray tracing algorithm, the propagation path of light at different time points is simulated to obtain the illumination distribution at different time points. Assume that the ray tracing result is ,in Indicates The illumination distribution at each time point is compared and analyzed in time and space between the time-varying theoretical illumination distribution atlas and the multi-station multimodal image dataset to identify the difference between the illumination distribution and the actual imaging results and generate a dynamic illumination error compensation coefficient matrix. Error compensation coefficient matrix Used to correct lighting errors and ensure imaging quality. According to the dynamic lighting error compensation coefficient matrix, the first multi-station collaborative lighting layout scheme is iteratively optimized across stations to generate a second multi-station collaborative lighting layout scheme. Through iterative optimization, the lighting uniformity and imaging effect are improved to obtain an optimized lighting layout. Based on the time series information of the second multi-station collaborative lighting layout scheme, the multi-station pan-tilt collaborative motion trajectory, and the multi-station multimodal image data set, dynamic dimming sequence and stroboscopic synchronization calculations are performed to generate a multimodal collaborative lighting control instruction set. The purpose of the dynamic dimming sequence and stroboscopic synchronization is to ensure that the light source can provide the best lighting effect at different time points and imaging conditions. Suppose the control instruction set is ,in Indicates The lighting control instructions for each time period are generated. The multi-modal collaborative lighting control instruction set is cross-station energy consumption balanced and instruction flow optimized to generate multi-station pan-tilt dynamic lighting control parameters. The purpose of energy consumption balancing is to optimize energy consumption and reduce energy waste while ensuring the lighting effect. Instruction flow optimization ensures that the lighting control instructions can be executed efficiently and maximizes the overall performance of the system.

[0108] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0109] (1) According to the dynamic lighting control parameters of the multi-station gimbal, the multi-station multimodal image dataset is corrected for illumination non-uniformity to obtain a corrected image set;

[0110] (2) Perform multi-scale and multi-directional feature extraction and cross-station matching on the corrected image set to obtain a feature correspondence matrix, and perform geometric transformation and precise registration based on the feature correspondence matrix to obtain a multi-station aligned image set;

[0111] (3) Perform multimodal data normalization on the multi-station aligned image set to obtain a standardized multimodal dataset, and construct a high-dimensional feature tensor of the item that integrates the station information based on the standardized multimodal dataset;

[0112] (4) Perform cross-station tensor decomposition and feature importance evaluation on the high-dimensional feature tensor of the item that integrates the workstation information to obtain the core feature representation of the item, and construct an initial item digital model containing the workstation detection information based on the core feature representation of the item;

[0113] (5) Perform multi-station semantic segmentation and comprehensive attribute mapping on the initial digital model of the object to obtain a comprehensive digital model of the object.

[0114] Specifically, the multi-station multi-modal image dataset is corrected for illumination non-uniformity using the dynamic illumination control parameters of the multi-station pan-tilt system, eliminating the image brightness and contrast differences caused by inconsistent lighting conditions and improving the overall image quality and consistency. Assume that the image dataset is ,in Indicates By applying dynamic lighting control parameters , each image is corrected, the formula is as follows:

[0115] ;

[0116] in, Represents the corrected image. Perform multi-scale and multi-directional feature extraction on the corrected image set. Identify key points and features in the image for subsequent matching and registration. Assume that the feature vector is ,in Indicates The features of each image. By performing multi-scale and multi-directional analysis on each image, key feature points are extracted. Cross-station matching is performed to obtain a feature correspondence matrix. Feature correspondence matrix Represents the feature matching relationship between different images, where Indicates The characteristics of the first image and the The correspondence between the features of the two images. Based on the feature correspondence matrix, geometric transformation and precise registration are performed. Assume that the geometric transformation matrix is ,in Indicates The geometric transformation of the images. Each image is registered by applying the geometric transformation matrix, as follows:

[0117] ;

[0118] in, Represents the image after registration. Perform multimodal data normalization on the multi-station aligned image set to eliminate the differences between different modalities and obtain a standardized multimodal dataset. Assume that the standardized dataset is , through normalization, the images of different modalities are converted into a unified representation. Based on the standardized multimodal dataset, a high-dimensional feature tensor of items integrating workstation information is constructed. Assume that the high-dimensional feature tensor is ,in Indicates The high-dimensional feature tensor of the item with integrated workstation information is decomposed into cross-workstation tensors and feature importance is evaluated to obtain the core feature representation of the item. The high-dimensional feature tensor is decomposed into low-dimensional sub-tensors to extract the core features. Assume that the tensor decomposition result is ,in Indicates core features. By evaluating the importance of each core feature, the most representative feature is selected to construct an initial digital model of the object containing the station detection information. Multi-station semantic segmentation and attribute comprehensive mapping are performed on the initial digital model of the object. The digital model is divided into different functional areas, and the attributes of each area are comprehensively mapped to form a complete object attribute map. Assume that the semantic segmentation result is ,in Indicates By integrating the inspection data and attribute information of different workstations, a comprehensive digital model of the object is obtained.

[0119] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0120] (1) Perform multi-scale feature decomposition and cross-station statistical analysis on the comprehensive digital model of the object to obtain the key feature distribution map of the object, and conduct a comprehensive evaluation based on the key feature distribution map of the object to obtain a multi-dimensional station inspection efficiency matrix;

[0121] (2) Perform multivariate correlation analysis on the multi-dimensional workstation detection efficiency matrix to obtain a dynamic relationship model between workstation movement and detection efficiency. Perform parameter testing based on the dynamic relationship model between workstation movement and detection efficiency to obtain performance bottleneck identification results.

[0122] (3) Based on the performance bottleneck identification results, an adaptive parameter sensitivity analysis is performed on the multi-station pan / tilt configuration scheme to obtain system optimization direction recommendations. The system optimization direction recommendations are then evaluated and ranked across stations to obtain a phased system optimization execution plan.

[0123] (4) Construct a multi-level workstation status prediction model based on the phased system optimization execution plan, and generate a multi-dimensional performance degradation trend chart through the multi-level workstation status prediction model;

[0124] (5) Perform dynamic threshold analysis and cumulative risk assessment on the multi-dimensional performance degradation trend chart to obtain the workstation-level maintenance warning indicators, and perform time-varying reliability analysis based on the workstation-level maintenance warning indicators to obtain the system maintenance warning strategy;

[0125] (6) Integrate the system maintenance warning strategy and the phased system optimization execution plan to obtain the multi-station pan-tilt system optimization plan and station maintenance warning information.

[0126] Specifically, multi-scale feature decomposition and cross-station statistical analysis are performed on the comprehensive digital model of the object to extract key features and generate a key feature distribution map of the object. Multi-scale feature decomposition is to extract the main features of the model by analyzing the digital model of the object at different scales. Assume that the digital model of the object is , multi-scale feature decomposition can be achieved through wavelet transform, expressed as:

[0127] ;

[0128] in, is the decomposed feature set, is the characteristic coefficient, is the wavelet basis function. By performing cross-station statistical analysis on these features, a distribution map of key features of items is generated. The distribution map of key features shows the distribution of features of items at different stations, which helps to identify key areas in the detection process. A comprehensive evaluation is performed based on the distribution map of key features of items to generate a multi-dimensional station detection efficiency matrix. This matrix reflects the efficiency of each station in detecting different features, expressed as:

[0129] ;

[0130] in, Indicates The workstation is testing The efficiency of the system when the feature is present. Multivariate correlation analysis is performed on the multi-dimensional workstation detection efficiency matrix to establish a dynamic relationship model between workstation motion and detection efficiency. By analyzing the relationship between the motion state of the workstation and the detection efficiency, the performance bottleneck of the system is identified. Assuming that the motion state of the workstation is , the detection efficiency is , the dynamic relationship model is expressed as:

[0131] ;

[0132] in, is the regression coefficient. Through parameter testing, the performance bottleneck is determined and the key factors affecting system performance are identified. According to the performance bottleneck identification results, the adaptive parameter sensitivity analysis of the multi-station PTZ configuration scheme is performed to evaluate the impact of different parameters on system performance and obtain system optimization direction recommendations. Assume that the parameter set is , the system performance is , the sensitivity analysis model is expressed as:

[0133] ;

[0134] in, Representation parameters About performance Based on these optimization suggestions, cross-station collaborative evaluation and sorting are carried out to form a phased system optimization execution plan. The optimization process is divided into multiple stages and implemented step by step to ensure a smooth transition and optimization effect of the system. Assume that the phased plan is , the goals and steps of each stage are determined through collaborative evaluation. Based on the phased system optimization execution plan, a multi-level station status prediction model is constructed. This model predicts future performance changes by analyzing the historical data and current status of each station. Assume that the station status is , the prediction model is:

[0135] ;

[0136] in, It's time The predicted value of the state at time is the regression coefficient. Through the multi-level station status prediction model, a multi-dimensional performance degradation trend chart is generated to show the change trend of system performance over time. Dynamic threshold analysis and cumulative risk assessment are performed on the multi-dimensional performance degradation trend chart to determine the station level maintenance warning indicator. Assume that the threshold is , the risk assessment is , the early warning indicator is expressed as:

[0137] ;

[0138] in, Indicates The workstation is Early warning indicators for each time period. According to the maintenance early warning indicators at the workstation level, time-varying reliability analysis is performed to obtain the system maintenance early warning strategy. Time-varying reliability analysis determines the best maintenance time and strategy by evaluating the reliability of the system in different time periods. Assume that the system reliability is The maintenance strategy is , expressed as:

[0139] ;

[0140] in, Indicates The system maintenance early warning strategy and phased system optimization execution plan are integrated to generate multi-station PTZ system optimization plan and station maintenance early warning information. By combining the optimization plan and maintenance strategy, the system can be ensured to maintain efficient operation and avoid sudden failures during the optimization process.

[0141] The above describes the multi-station pan-tilt control method in the embodiment of the present application. The following describes the multi-station pan-tilt control device in the embodiment of the present application. Figure 2In the embodiment of the present application, an embodiment of a multi-station pan-tilt control device includes:

[0142] The acquisition module 201 is used to acquire the object characteristic data of the target object and analyze and process it to obtain a multi-station pan-tilt configuration scheme and a station detection area allocation strategy;

[0143] A planning module 202 is used to plan a collaborative motion path based on a multi-station pan-tilt configuration scheme and a station detection area allocation strategy to obtain a multi-station pan-tilt collaborative motion trajectory;

[0144] The scanning module 203 is used to control the multi-station pan-tilt platform to synchronously perform multi-modal imaging scanning on the target object according to the coordinated motion trajectory of the multi-station pan-tilt platform to obtain a multi-station multi-modal image data set;

[0145] The analysis module 204 is used to perform comprehensive scene analysis on the multi-station multi-modal image data set to obtain the multi-station PTZ dynamic lighting control parameters;

[0146] A fusion module 205 is used to perform cross-station data fusion on the multi-station multimodal image data set based on the multi-station pan-tilt dynamic lighting control parameters to obtain a comprehensive digital model of the object;

[0147] The optimization module 206 is used to perform self-optimization analysis on the comprehensive digital model of the object and the multi-station detection process data to obtain the multi-station pan-tilt system optimization plan and station maintenance warning information.

[0148] Through the cooperation of the above components, by acquiring and analyzing the characteristic data of the target object, the surface of the object can be analyzed and adaptively segmented, and the detection sub-areas can be effectively allocated. The detection area is optimized and allocated by using multi-target clustering analysis and load balancing strategy, which improves the detection efficiency of the workstation and the operation efficiency of the overall system. Based on the multi-station pan-tilt configuration scheme and the detection area allocation strategy, collaborative motion path planning is carried out to obtain efficient collaborative motion trajectories. Through multi-station spatiotemporal conflict detection and dynamic priority allocation, efficient collaborative motion between multiple stations is achieved, reducing interference and conflict between stations. The multi-station pan-tilt is controlled to perform multi-modal imaging scanning synchronously to obtain high-precision multi-modal image data sets and improve imaging quality. Adaptive dynamic focus adjustment and real-time correction and denoising processing are used to obtain high-definition image groups to ensure the accuracy and integrity of image data. Based on comprehensive scene analysis, the lighting control parameters of the pan-tilt are dynamically adjusted, the lighting layout and light distribution are optimized, and the imaging quality is improved. Through dynamic ray tracing simulation and lighting error compensation, precise lighting control is achieved, reducing the impact of uneven lighting on imaging. Perform illumination correction and feature matching on multi-station multimodal image data to achieve cross-station data fusion and build a comprehensive digital model of the object. Use multi-scale feature extraction and geometric transformation to ensure the accuracy of data fusion and the integrity of model construction, and provide a high-precision digital representation of the object. Based on multi-station detection process data and object digital models, perform self-optimization analysis, propose system optimization solutions, and improve the operating efficiency of the system. Generate maintenance warning information through multi-level station status prediction and dynamic threshold analysis to prevent potential failures and ensure the reliability and stability of the system.

[0149] The present application also provides a multi-station pan-tilt control device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the multi-station pan-tilt control method in the above-mentioned embodiments.

[0150] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-station pan-tilt control method.

[0151] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent item, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software item. The computer software item is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0153] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-position PTZ control method, characterized in that: The method comprises: The object characteristic data of the target object is obtained and analyzed and processed to obtain a multi-station pan-tilt configuration scheme and a station detection area allocation strategy; specifically including: obtaining the object characteristic data of the target object, and based on the object characteristic data, performing feature analysis and adaptive area segmentation on the surface of the target object to obtain multiple detection sub-regions; performing multi-dimensional feature extraction on each detection sub-region to obtain a sub-region feature vector set, and based on the sub-region feature vector set, performing detection difficulty assessment and priority sorting on each detection sub-region to obtain a regional detection complexity index; based on the regional detection complexity index, performing multi-target clustering analysis on the multiple detection sub-regions to obtain an initial station allocation scheme, and performing station load balancing and detection efficiency optimization calculation on the initial station allocation scheme to obtain a station detection area allocation strategy; based on the station detection area allocation strategy, performing detection requirements and constraint analysis on each station pan-tilt to obtain station function requirement specifications, and based on the station function requirement specifications, performing pan-tilt hardware configuration and parameter optimization on each station pan-tilt to obtain a multi-station pan-tilt configuration scheme; Based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy, collaborative motion path planning is performed to obtain a multi-station pan-tilt collaborative motion trajectory; According to the collaborative motion trajectory of the multi-station pan-tilt platform, the multi-station pan-tilt platform is controlled to synchronously perform multi-modal imaging scanning on the target object to obtain a multi-station multi-modal image data set; Performing comprehensive scene analysis on the multi-station multimodal image data set to obtain dynamic lighting control parameters of the multi-station pan / tilt platform; Based on the multi-station pan-tilt dynamic lighting control parameters, cross-station data fusion is performed on the multi-station multimodal image data set to obtain a comprehensive digital model of the object; A self-optimizing analysis is performed on the comprehensive digital model of the object and the multi-station detection process data to obtain a multi-station pan / tilt system optimization solution and station maintenance warning information.

2. The multi-position PTZ control method according to claim 1, characterized in that: The collaborative motion path planning based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy to obtain the multi-station pan-tilt collaborative motion trajectory includes: According to the multi-station pan-tilt configuration scheme, kinematic modeling is performed on each station pan-tilt to obtain a kinematic model of the station pan-tilt; Based on the workstation detection area allocation strategy, adaptive coverage path planning is performed to obtain an initial detection path set, and multi-station spatiotemporal conflict detection and analysis is performed on the initial detection path set to obtain a potential conflict point set; According to the potential conflict point set and the detection area priority, dynamically assigning priorities and adjusting timings for the initial detection path set to obtain an initial collaborative path solution; Performing multi-station parallel kinematic inverse solution calculation on the initial collaborative path scheme to obtain a joint motion sequence, and performing polynomial interpolation and spline curve fitting based on the joint motion sequence to obtain a smooth motion trajectory; The smooth motion trajectory is subjected to a spatiotemporal coordination analysis among multiple workstations to obtain a workstation coordination index and a coordination efficiency score, and global iterative optimization and adjustment are performed based on the workstation coordination index and the coordination efficiency score to obtain a multi-workstation highly coordinated motion trajectory.

3. The multi-position PTZ control method according to claim 2, characterized in that: According to the multi-station pan-tilt coordinated motion trajectory, the multi-station pan-tilt is controlled to synchronously perform multi-modal imaging scanning on the target object to obtain a multi-station multi-modal image data set, including: Performing time series decomposition and interpolation on the collaborative motion trajectory of the multi-station pan-tilt platform to obtain a high-precision discrete time point position sequence of each station pan-tilt platform; According to the high-precision discrete time point position sequence and the workstation detection area allocation strategy, parameter optimization calculation is performed on the multimodal imaging device of each workstation pan / tilt to obtain an optimal imaging parameter set; Based on the optimal imaging parameter set and the multi-station pan-tilt configuration scheme, a cross-station synchronous trigger timing design is performed on the multimodal imaging devices of the pan-tilts of each station to obtain a globally optimized multimodal imaging synchronous control instruction; The globally optimized multi-modal imaging synchronization control instructions are processed in parallel and task-assigned to obtain a coordinated parallel control instruction stream, and a real-time motion error compensation calculation is performed on the drive system of each station pan-tilt according to the coordinated parallel control instruction stream and the kinematic model of the station pan-tilt to obtain a target real-time position control signal; Based on the target real-time position control signal, adaptive dynamic focusing adjustment is performed on the imaging system of each workstation pan / tilt to obtain a multi-scale focal plane compensation parameter, and according to the multi-scale focal plane compensation parameter and the workstation coordination index, real-time correction and denoising processing is performed on the acquired original multimodal image data to obtain a high-definition image group; Performing multi-scale and multi-directional feature extraction on the high-definition image group to obtain a multi-dimensional feature vector set, and performing cross-station area matching and spatial registration based on the multi-dimensional feature vector set and the station detection area allocation strategy to obtain a multi-station spatial alignment image set; Time series correlation analysis and data completion are performed on the multi-station spatially aligned image set to obtain a multi-station multimodal image dataset.

4. The multi-position PTZ control method according to claim 3, characterized in that: The comprehensive scene analysis is performed on the multi-station multi-modal image data set to obtain the multi-station pan-tilt dynamic lighting control parameters, including: Performing modal separation and multi-scale decomposition on the multi-station multi-modal image data set to obtain a multi-level image pyramid structure, and performing adaptive local contrast analysis based on the multi-level image pyramid structure to obtain a multi-modal fused target object surface reflection characteristic map; Based on the multi-modal fused target object surface reflection characteristic map, a multi-dimensional lighting demand assessment is performed to obtain a regional lighting parameter demand matrix, and the regional lighting parameter demand matrix is ​​analyzed to obtain a composite regional group; According to the composite area group, globally optimize the light source type and position of the multi-spectral lighting equipment of each workstation pan / tilt to obtain a first multi-workstation collaborative lighting layout solution; Based on the first multi-station collaborative lighting layout scheme and the multi-station pan-tilt collaborative motion trajectory, a dynamic ray tracing simulation is performed to obtain a time-varying theoretical illumination distribution atlas, and a spatiotemporal comparative analysis is performed between the time-varying theoretical illumination distribution atlas and the multi-station multimodal image data set to obtain a dynamic illumination error compensation coefficient matrix; According to the dynamic illumination error compensation coefficient matrix, performing cross-station iterative optimization on the first multi-station collaborative lighting layout solution to obtain a second multi-station collaborative lighting layout solution; Based on the second multi-station collaborative lighting layout scheme, the multi-station pan-tilt collaborative motion trajectory and the time series information of the multi-station multimodal image data set, dynamic dimming sequence and stroboscopic synchronization calculation are performed to obtain a multimodal collaborative lighting control instruction set; The multi-modal collaborative lighting control instruction set is subjected to cross-station energy consumption balancing and instruction flow optimization to obtain multi-station pan-tilt dynamic lighting control parameters.

5. The multi-station PTZ control method according to claim 4, characterized in that: Based on the multi-station pan-tilt dynamic lighting control parameters, cross-station data fusion is performed on the multi-station multimodal image data set to obtain a comprehensive digital model of the object, including: According to the multi-station pan-tilt dynamic lighting control parameters, performing illumination non-uniformity correction on the multi-station multimodal image data set to obtain a corrected image set; Performing multi-scale and multi-directional feature extraction and cross-station matching on the corrected image set to obtain a feature correspondence matrix, and performing geometric transformation and precise registration based on the feature correspondence matrix to obtain a multi-station aligned image set; Performing multimodal data normalization processing on the multi-station aligned image set to obtain a standardized multimodal data set, and constructing a high-dimensional feature tensor of an item integrating station information according to the standardized multimodal data set; Performing cross-station tensor decomposition and feature importance evaluation on the high-dimensional feature tensor of the item fused with the station information to obtain a core feature representation of the item, and constructing an initial item digital model containing the station detection information based on the core feature representation of the item; The initial digital model of the object is subjected to multi-station semantic segmentation and comprehensive attribute mapping to obtain a comprehensive digital model of the object.

6. The multi-position PTZ control method according to claim 5, characterized in that: The self-optimization analysis of the comprehensive digital model of the object and the multi-station detection process data to obtain the multi-station pan / tilt system optimization plan and station maintenance warning information includes: Perform multi-scale feature decomposition and cross-station statistical analysis on the comprehensive digital model of the object to obtain a key feature distribution map of the object, and perform a comprehensive evaluation based on the key feature distribution map of the object to obtain a multi-dimensional station detection efficiency matrix; Performing multivariate correlation analysis on the multi-dimensional workstation detection efficiency matrix to obtain a workstation movement-detection efficiency dynamic relationship model, and performing parameter testing based on the workstation movement-detection efficiency dynamic relationship model to obtain a performance bottleneck identification result; According to the performance bottleneck identification result, an adaptive parameter sensitivity analysis is performed on the multi-station pan / tilt configuration scheme to obtain system optimization direction suggestions, and cross-station collaborative evaluation and sorting are performed on the system optimization direction suggestions to obtain a phased system optimization execution plan; Building a multi-level workstation status prediction model based on the phased system optimization execution plan, and generating a multi-dimensional performance decay trend graph through the multi-level workstation status prediction model; Performing dynamic threshold analysis and cumulative risk assessment on the multi-dimensional performance degradation trend graph to obtain a workstation-level maintenance warning indicator, and performing time-varying reliability analysis based on the workstation-level maintenance warning indicator to obtain a system maintenance warning strategy; The system maintenance warning strategy and the phased system optimization execution plan are integrated to obtain a multi-station pan / tilt system optimization solution and station maintenance warning information.

7. A multi-station PTZ control device, characterized in that: The device is used to execute the multi-station pan-tilt control method according to any one of claims 1 to 6, comprising: An acquisition module is used to acquire the object characteristic data of the target object and analyze and process it to obtain a multi-station pan-tilt configuration scheme and a station detection area allocation strategy; specifically comprising: acquiring the object characteristic data of the target object, and performing feature analysis and adaptive area segmentation on the surface of the target object according to the object characteristic data to obtain multiple detection sub-regions; performing multi-dimensional feature extraction on each detection sub-region to obtain a sub-region feature vector set, and performing detection difficulty assessment and priority sorting on each detection sub-region according to the sub-region feature vector set to obtain a regional detection complexity index; based on the regional detection complexity index, performing multi-objective clustering analysis on the multiple detection sub-regions to obtain an initial station allocation scheme, and performing station load balancing and detection efficiency optimization calculation on the initial station allocation scheme to obtain a station detection area allocation strategy; according to the station detection area allocation strategy, performing detection requirements and constraint condition analysis on each station pan-tilt to obtain station function requirement specifications, and performing pan-tilt hardware configuration and parameter optimization on each station pan-tilt based on the station function requirement specifications to obtain a multi-station pan-tilt configuration scheme; A planning module, used to plan a collaborative motion path based on the multi-station pan-tilt configuration scheme and the station detection area allocation strategy to obtain a multi-station pan-tilt collaborative motion trajectory; A scanning module, used for controlling the multi-station pan-tilt table to synchronously perform multi-modal imaging scanning on the target object according to the coordinated motion trajectory of the multi-station pan-tilt table, so as to obtain a multi-station multi-modal image data set; An analysis module, used to perform comprehensive scene analysis on the multi-station multi-modal image data set to obtain dynamic lighting control parameters of the multi-station pan / tilt platform; A fusion module, used for performing cross-station data fusion on the multi-station multimodal image data set based on the multi-station pan-tilt dynamic lighting control parameters to obtain a comprehensive digital model of the object; The optimization module is used to perform self-optimization analysis on the comprehensive digital model of the object and the multi-station detection process data to obtain the multi-station pan-tilt system optimization plan and station maintenance early warning information.

8. A multi-station PTZ control device, characterized in that: The multi-station PTZ control device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the multi-station pan-tilt control device executes the multi-station pan-tilt control method as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the multi-station pan-tilt control method according to any one of claims 1 to 6 is implemented.

Images