Tower crane control system based on real-time video panoramic stitching technology

Abstract

The invention discloses a tower crane control system based on real-time video panoramic stitching technology, which relates to the technical field of panoramic stitching and comprises a data acquisition module, which is used for acquiring system response data and image data of real-time video panoramic stitching under different input resolutions; a comprehensive evaluation module, which is used for constructing a resolution impact evaluation model based on a long-term and short-term neural network according to the system response data and image sensory data; an input data volume judgment module, which is used for constructing an input resolution-stitching effect curve diagram according to the output of the resolution impact evaluation model under different input resolutions; a discrimination constraint module, which is used for performing data analysis and constructing discrimination constraint conditions based on hardware data and artificial preset restrictions; a resolution input / output module, which is used for taking the peak value of the resolution-stitching effect curve diagram of the discrimination constraint condition as the optimal input resolution and performing panoramic stitching; and the problem of delay and quality imbalance of panoramic stitching caused by poor input resolution is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of panoramic stitching, and more specifically, to a tower crane control system based on real-time video panoramic stitching technology. Background Art

[0002] Panoramic stitching technology is a technology that combines multiple overlapping images or video clips into a panoramic image with a wide field of view. It combines images from multiple perspectives into a seamless panoramic view through steps such as image acquisition, geometric calibration, feature matching, image registration and fusion. In the tower crane control system, this technology can generate a panoramic view of the tower crane's surroundings in real time, helping operators to have a more comprehensive understanding of the surrounding conditions and improve the safety and accuracy of operations.

[0003] For example, the invention patent with publication number: CN108975165B discloses a tower crane monitoring system and method and a tower crane, which is installed on the tower crane through an image acquisition device, and is used to shoot the operation of the tower crane to obtain the operation image of the tower crane; a receiving device is used to receive the working data of the tower crane and the status information of the image acquisition device; and a control device is used to adjust the shooting angle of the image acquisition device according to the working data and the status information. The present invention installs an image acquisition device on the tower crane to shoot the operation of the tower crane, and adjusts the shooting angle of the image acquisition device according to the working data of the tower crane and the status information of the image acquisition device, so as to make full use of the detection function of the camera to achieve the purpose of safety control.

[0004] For example, the tower crane monitoring and early warning method, system and tower crane disclosed in the invention patent with publication number: CN102583181A, by: respectively acquiring images of obstacles collected by the first imaging device and the second imaging device, wherein the first imaging device and the second imaging device are both located on the tower arm of the tower crane; determining the distance between the obstacle and the tower arm of the tower crane according to the image, the positional relationship information of the first imaging device and the second imaging device; and initiating an alarm when the distance is less than a preset threshold. In the present invention, the image of the obstacle is collected by two imaging devices to determine the distance between the obstacle and the tower arm of the tower crane, so as to monitor whether the tower arm and the hook will collide with the obstacle. Starting from the obstacle to monitor the tower crane, the tower crane collision monitoring level is improved, the occurrence of tower crane collision accidents is prevented, and the tower crane operates safely and efficiently.

[0005] The above disclosed technical solutions have at least the following technical problems: the tower crane control system needs to process image and video data in real time to ensure safe and accurate operation, but the existing panoramic stitching technology may introduce delays, affecting the real-time operation. Such delays may cause the operator to be unable to obtain an accurate panoramic view in real time, thereby affecting operational decisions, increasing the risk of accidents, reducing operational efficiency, and possibly causing the tower crane to collide with surrounding objects, or reducing the accuracy and safety of tower crane operation.

[0006] In view of the above problems, the present invention proposes a solution. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a tower crane control system based on real-time video panoramic stitching technology, which solves the problem of poor input resolution, resulting in delay and quality imbalance of panoramic stitching through panoramic stitching technology evaluation.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] In a preferred embodiment, it includes a data acquisition module, a comprehensive evaluation module, an input data volume judgment module, a discrimination constraint module and a resolution input / output module; the data acquisition module is used to obtain system response data and image data of real-time video panoramic stitching at different input resolutions; the comprehensive evaluation module is used to construct a resolution impact evaluation model based on the long-term and short-term neural network according to the system response data and image sensory data; the input data volume judgment module is used to construct an input resolution-stitching effect curve diagram according to the output of the resolution impact evaluation model at different input resolutions; the discrimination constraint module is used to perform data analysis and construct discrimination constraint conditions based on hardware data and manually preset restrictions; the resolution input / output module is used to cut the input resolution-stitching effect curve diagram according to the discrimination constraint conditions, and take the peak value of the cut resolution-stitching effect curve diagram as the optimal input resolution, and perform panoramic stitching.

[0010] In a preferred embodiment, the system response data includes a response rate evaluation coefficient; the image data includes an image quality impact coefficient and an image stitching event impact evaluation coefficient.

[0011] In a preferred embodiment, a specific method for obtaining the response rate evaluation coefficient is as follows: at different input resolutions, the time required for the system to receive a video frame, complete processing, and generate a stitching result is obtained as the delay time; for the same resolution, multiple measurements are performed, abnormal delay times are removed, and the average delay time is calculated; the number of frames processed by the system at different input resolutions and the total time required for processing are obtained, and the system video processing rate is calculated; an initial experience diminishing effect model is set based on a power function model according to the average delay time and the system video processing rate; several delay time points are set for user experience testing, and at each delay time point, user experience rating data is collected; based on the experience rating data, the undetermined parameters of the initial experience diminishing effect model are optimized based on the nonlinear least squares method to obtain a calculation model for the response rate evaluation coefficient, wherein the undetermined parameters include the maximum value of the user experience, the rate of decrease of the experience with increasing delay, and the sensitivity coefficient of delay to the diminishing experience.

[0012] In a preferred embodiment, the specific method for obtaining the image quality influence coefficient is as follows: a positive stitching deformation reference value is set for the positive stitching deformation generated by panoramic stitching of panoramic image intake points, and a negative stitching deformation reference value is set for the negative stitching deformation; a positive stitching deformation less than the positive stitching deformation reference value is set as an acceptable positive stitching deformation, and a positive stitching deformation greater than the positive stitching deformation reference value is set as a potential positive stitching deformation; a negative stitching deformation less than the negative stitching deformation reference value is set as a potential negative stitching deformation, The negative stitching deformation that is greater than the negative stitching deformation reference value is set as an acceptable negative stitching deformation; the positive stitching deformation value and the negative stitching deformation value of the panoramic image intake point are obtained, and the positive stitching deformation value is compared with the positive stitching deformation reference value, a positive stitching deformation set is established for the positive stitching deformation values ​​of all potential positive stitching deformations, and a negative stitching deformation set is established for the negative stitching deformation values ​​of all potential negative stitching deformations; data analysis is performed based on the positive stitching deformation set and the negative stitching deformation set, and the image quality influence coefficient is calculated.

[0013] In a preferred embodiment, a specific method for obtaining the image stitching event impact assessment coefficient is as follows: based on the historical impact event types of image stitching at different resolutions, a set of impact events for image stitching at different resolutions is obtained, wherein the set of impact events for image stitching at different resolutions includes various abnormal events of stitching and impact events caused by human factors at different resolutions, as well as event effect ratings; various types of image stitching at different resolutions within the set of impact events for image stitching at different resolutions are assigned preset impact assessment values ​​based on the event effect ratings; a regression model is constructed based on the set of image stitching at different resolutions and the corresponding impact assessment values ​​to generate an image stitching event impact assessment coefficient.

[0014] In a preferred embodiment, the output of the angular resolution impact assessment model for different input resolutions is used to construct an input resolution-splicing effect curve graph, specifically: the resolution impact assessment coefficient of the output of the input resolution under different settings is obtained from the resolution impact assessment model under different input resolutions; a rectangular coordinate system is established, with the horizontal axis marked as the input resolution and the vertical axis as the resolution impact assessment coefficient; the resolution impact assessment coefficient corresponding to each input resolution is marked on the rectangular coordinate system to form data points; and the data points are connected to form an input resolution-splicing effect curve graph.

[0015] In a preferred embodiment, the method for obtaining the discrimination constraint conditions is as follows: collect the system's hardware performance data, the hardware performance data including processing power, storage limitations and bandwidth, and manually set image quality requirements, response time and stability standards; by establishing a performance model and a constraint model, integrate the performance constraints and quality standards to construct the discrimination constraint conditions.

[0016] In a preferred embodiment, the input resolution-splicing effect curve graph is cut according to the discrimination constraint conditions, specifically: the curves in the input resolution-splicing effect curve graph that do not meet the discrimination constraint conditions are removed, and the input resolution-splicing effect curve graph that meets the discrimination constraint conditions is left.

[0017] The technical effects and advantages of the tower crane control system based on real-time video panoramic stitching technology of the present invention are as follows:

[0018] 1. The present invention integrates modules such as data acquisition, comprehensive evaluation, input data volume judgment, discrimination constraints, and resolution input and output, and realizes comprehensive optimization of real-time video panoramic stitching performance under different input resolutions. The system can accurately quantify the impact of response rate, image quality, and stitching events, and then select the optimal input resolution that meets system performance and user experience requirements. The response rate evaluation coefficient effectively reduces system latency and improves real-time performance; the image quality impact coefficient ensures image clarity and visual effects; and the image stitching event impact evaluation coefficient reduces visual defects during the stitching process. Combining these evaluation coefficients, the system can scientifically and rationally optimize resolution selection, improve operational accuracy and safety, and enhance user experience.

[0019] 2. The present invention uses a comprehensive evaluation module, an input data volume judgment module, a discrimination constraint module, and a resolution input / output module, and uses a long-term and short-term neural network to construct a resolution impact evaluation model to optimize the input resolution selection of the tower crane control system; this embodiment integrates system response data, image quality impact data, and splicing event impact data into a resolution impact evaluation coefficient, and based on this, establishes an input resolution-splicing effect curve chart, effectively converting performance and quality standards into an operational decision basis. Furthermore, the discrimination constraint module combines hardware performance and preset standards to screen the curve chart to ensure that only resolutions that meet the requirements are retained, thereby optimizing the panoramic splicing effect and improving system performance and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural schematic diagram of a tower crane control system based on real-time video panoramic stitching technology of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Embodiment 1, Figure 1 The invention provides a tower crane control system based on real-time video panoramic stitching technology, which includes a data acquisition module, a comprehensive evaluation module, an input data volume judgment module, a judgment constraint module and a resolution input and output module.

[0023] A data acquisition module, used to acquire system response data and image data of real-time video panoramic stitching at different input resolutions;

[0024] A comprehensive evaluation module for building a resolution impact evaluation model based on long-term and short-term neural networks according to system response data and image sensory data;

[0025] The input data volume judgment module is used to construct an input resolution-stitching effect curve diagram according to the output of the resolution impact assessment model under different input resolutions;

[0026] The discrimination constraint module is used to perform data analysis and construct discrimination constraint conditions based on hardware data and manually preset restrictions;

[0027] The resolution input and output module is used to cut the input resolution-stitching effect curve according to the judgment constraint conditions, and use the peak value of the cut resolution-stitching effect curve as the optimal input resolution, and perform panoramic stitching.

[0028] In this embodiment, the data acquisition module is used to acquire system response data and image data of real-time video panoramic stitching at different input resolutions. In this embodiment, the system response data includes a response rate evaluation coefficient; the image data includes an image quality impact coefficient and an image stitching event impact evaluation coefficient.

[0029] Furthermore, in this embodiment, the response rate evaluation coefficient is an indicator used to quantify the real-time video processing capability of the tower crane control system at different input resolutions; specifically, the coefficient measures the response time and delay of the system when processing and splicing video data, and helps determine the impact of video resolution on system performance; the higher the response rate evaluation coefficient, the faster the system can respond and process video data at this resolution, thereby reducing delays and improving real-time performance. Therefore, the coefficient can be used to effectively evaluate and select the optimal input resolution to optimize the real-time performance of panoramic splicing and ensure the safety and operating accuracy of the tower crane control system.

[0030] Furthermore, analyzing the response rate evaluation coefficient has the following advantages for determining the optimal input resolution:

[0031] Accurate real-time evaluation: The response rate evaluation coefficient directly measures the response time and delay of the system when processing video data at different input resolutions. It can accurately evaluate the real-time performance of video processing, thereby selecting the best input resolution that reduces delay while ensuring system performance;

[0032] Optimize operational efficiency: By quantifying the response rate at different resolutions, it can help determine a balance point that can ensure image quality while achieving low latency, optimize the operational efficiency of the tower crane control system, and avoid affecting the operator's real-time decision-making due to high latency;

[0033] Improve safety: Selecting the best input resolution can effectively reduce potential risks caused by delays, such as collisions between the crane and surrounding objects, or decreased operating accuracy, thereby improving the overall safety of the system;

[0034] Reduce system burden: Through the response rate evaluation coefficient, unnecessary high-resolution inputs can be identified and excluded, thereby reducing the burden of calculation and data processing and improving system stability and reliability;

[0035] Data-driven decision support: The response rate assessment coefficient provides a data-based decision support tool, making the process of selecting the best input resolution more scientific and reasonable, rather than relying solely on experience or subjective judgment;

[0036] Improve user experience: Optimized resolution selection can improve image smoothness and clarity while maintaining low latency, thereby enhancing the operator's visual experience and operating comfort;

[0037] In summary, analyzing the response rate evaluation coefficient is of great significance for determining the optimal input resolution.

[0038] Furthermore, the specific method for obtaining the response rate evaluation coefficient is as follows:

[0039] Under different input resolutions, the time required for the system to receive a video frame, complete processing, and generate a stitching result is obtained as the delay time;

[0040] For the same resolution, multiple measurements are performed, abnormal delay times are removed, and the average delay time is calculated;

[0041] Obtain the number of frames processed by the system at different input resolutions and the total time required for processing, and calculate the system video processing rate;

[0042] An initial experience diminishing effect model is set based on a power function model according to the average delay time and system video processing rate;

[0043] Set several delay time points to conduct user experience testing, and collect user experience rating data at each delay time point;

[0044] According to the experience rating data, the undetermined parameters of the initial experience diminishing effect model are optimized based on the nonlinear least squares method to obtain a calculation model for the response rate evaluation coefficient. The undetermined parameters include the maximum value of the user experience, the rate of decrease of the experience with increasing delay, and the sensitivity coefficient of delay to the diminishing experience.

[0045] The specific calculation formula of the average delay time is as follows:

[0046]

[0047] The specific calculation formula of the error function of the present application included in the nonlinear least squares method is as follows:

[0048]

[0049] The specific calculation formula of the response rate evaluation coefficient is as follows:

[0050]

[0051] In the formula, is the average delay time, is the delay time of the ith measurement, To measure the delay times at the same input resolution, is the error function of the present application included in the nonlinear least squares method, The amount of user experience rating data used for optimization, The label for the user's experience rating data. Provides a user experience rating data value. is the response rate evaluation coefficient.

[0052] It should be noted that the maximum user experience refers to the theoretical maximum value of the response rate evaluation coefficient when the delay time is zero. In practical applications, this value represents the best user experience of the system under ideal conditions, that is, the maximum experience level felt by users when there is no delay or other negative impact. It is a benchmark for measuring system performance and experience quality, and is usually used to compare changes in user experience under different delay conditions.

[0053] It should be noted that as the delay time increases, the response rate evaluation coefficient will decrease in the form of a power function, which means that the rate of decline in user experience accelerates with the increase in delay; the decreasing speed is controlled by the rate of decline in experience with increasing delay and the value of the sensitivity coefficient of delay to the decrease in experience, where the sensitivity coefficient of delay to the decrease in experience adjusts the sensitivity of delay to experience, and the rate of decline in experience with increasing delay determines the steepness of the decline curve of experience with increasing delay; this decreasing effect reflects that in actual operation, the impact of increasing delay on user experience is gradually significant, that is, the greater the delay, the faster the experience decreases.

[0054] The image quality impact coefficient is used to evaluate the impact of image quality at different input resolutions on the overall system performance to help determine the optimal input resolution; this coefficient quantifies the impact of image resolution on image clarity, detail retention and visual effects, and is usually determined by comparing the clarity, color accuracy and detail sharpness of images at different resolutions; a high image quality impact coefficient means that a higher resolution provides better image quality, while a lower coefficient may indicate that the resolution is not sufficient to meet the requirements; by comprehensively considering the image quality impact coefficient, the resolution selection can be optimized to balance image clarity and system performance, thereby achieving the best visual effects and operational efficiency.

[0055] Furthermore, analyzing the image quality impact coefficient has the following advantages for determining the optimal input resolution:

[0056] Objective evaluation of image clarity: The image quality impact coefficient can accurately quantify the clarity and detail retention of images at different resolutions, helping to evaluate the actual impact of each resolution on image quality. Through this objective evaluation, the optimal resolution in terms of image detail and visual effect can be selected.

[0057] Balance visual effects and system performance: The image quality impact factor provides a specific indicator of the impact of resolution on image quality, so that while pursuing high visual quality, the system processing capacity and response speed can be comprehensively considered; this ensures that while improving image quality, system resources will not be over-consumed, affecting overall performance;

[0058] Optimize user experience: By analyzing the image quality impact factor, you can select the resolution that strikes the best balance between visual effects and system performance, thereby improving the user's visual experience. Ensuring that the image reaches an ideal level in terms of detail, color, and contrast helps improve user satisfaction and operational efficiency;

[0059] Support dynamic adjustment: Under different operating environments or task requirements, the image quality impact factor can be used to adjust the input resolution in real time to meet the current quality requirements and system load; this can achieve flexible resolution optimization to adapt to different usage scenarios and real-time requirements;

[0060] Data-driven decision making: The image quality impact factor provides a scientific basis based on actual data to determine the optimal input resolution. This data-driven approach reduces the bias of subjective judgment and makes the resolution selection process more accurate and reliable.

[0061] In summary, analyzing the image quality influence coefficient is of great significance for determining the optimal input resolution.

[0062] The specific method for obtaining the image quality influence coefficient is as follows:

[0063] The positive stitching deformation generated by panoramic stitching through the panoramic image intake points sets the positive stitching deformation reference value, and the negative stitching deformation sets the negative stitching deformation reference value. The positive stitching deformation less than the positive stitching deformation reference value is set as an acceptable positive stitching deformation, and the positive stitching deformation greater than the positive stitching deformation reference value is set as a potential positive stitching deformation; the negative stitching deformation less than the negative stitching deformation reference value is set as a potential negative stitching deformation, and the negative stitching deformation greater than the negative stitching deformation reference value is set as an acceptable negative stitching deformation;

[0064] Acquire positive stitching deformation values ​​and negative stitching deformation values ​​of the panoramic image intake point, compare the positive stitching deformation value with the positive stitching deformation reference value, establish a positive stitching deformation set with the positive stitching deformation values ​​of all hidden positive stitching deformations, and establish a negative stitching deformation set with the negative stitching deformation values ​​of all hidden negative stitching deformations;

[0065] Data analysis is performed based on the positive splicing deformation set and the negative splicing deformation set, and the image quality impact coefficient is calculated.

[0066] Calculate the image quality impact coefficient. The specific calculation formula is as follows:

[0067]

[0068] In the formula, is the image quality influence coefficient, is the positive splicing deformation value in the positive splicing deformation set, h is the number of the positive splicing deformation value in the positive splicing deformation set, is the negative splicing deformation value in the negative splicing deformation set, k is the number of the positive splicing deformation value in the positive splicing deformation set, , The time period for obtaining the forward stitching deformation value of the panoramic image intake point within the P time interval and the Q time interval, , The time period for obtaining negative stitching deformation values ​​of the panoramic image intake point within the P time and Q time interval.

[0069] It should be noted that positive stitching deformation refers to the magnification effect of objects or textures in the stitching area due to the expansion or stretching of the image during the panoramic stitching process; negative stitching deformation refers to the reduction effect of objects or textures in the stitching area due to the contraction or compression of the image; both types of deformation may affect the visual quality of the image, positive deformation makes the image appear larger or distorted, and negative deformation makes the image appear smaller or incomplete; in order to evaluate the optimal input resolution, it is necessary to set reference values ​​for positive and negative stitching deformations, compare the deformation values ​​obtained in the actual stitching process, and classify them as acceptable or potential deformations, thereby quantifying the image quality impact coefficient to ensure that the stitching effect meets the quality standards.

[0070] The image stitching event impact assessment coefficient is used to quantify and evaluate the impact of various events occurring during the panoramic stitching process on image quality and system performance, so as to help determine the optimal input resolution; this coefficient comprehensively considers the impact of stitching errors, distortion, ghosting or other visual defects that occur during the image stitching process on the overall image quality; by analyzing the frequency and severity of these stitching events, the stability and accuracy of the stitching effect at different resolutions can be determined, thereby selecting the optimal input resolution that can minimize these negative impacts.

[0071] Furthermore, the image stitching event impact assessment coefficient has the following advantages for determining the optimal input resolution:

[0072] Accurate identification of stitching issues: This factor can accurately identify and quantify specific events in the stitching process, such as stitching errors, distortion or ghosting, and provide a detailed analysis of the impact of these issues; this helps to understand potential problems that occur at different resolutions and choose a resolution that can effectively reduce these problems;

[0073] Improve stitching quality: By evaluating the impact of stitching events, the input resolution can be optimized to reduce visual defects and errors in the stitching process, thereby improving the overall quality and accuracy of the final image;

[0074] Optimize system performance: This factor comprehensively considers the impact of stitching events on system performance, helping to balance resolution selection and system resource consumption, thereby improving image quality while avoiding excessive consumption of computing and storage resources;

[0075] Improve user experience: Reducing the occurrence of stitching events can significantly improve the user's visual experience and satisfaction. The evaluation coefficient can guide the selection of the optimal resolution to ensure that the image stitching process is smooth and without obvious visual defects;

[0076] Data-driven decision support: By analyzing the impact assessment coefficient of splicing events, scientific and reasonable resolution selection decisions can be made based on data, avoiding errors caused by subjective judgment, making the selection process more accurate and reliable;

[0077] In summary, analyzing the impact assessment coefficient of image stitching events is of great significance for determining the optimal input resolution.

[0078] The specific method for obtaining the image splicing event impact assessment coefficient is as follows:

[0079] Based on the historical event types of image splicing at different resolutions, a set of events affecting image splicing at different resolutions is obtained, wherein the set of events affecting image splicing at different resolutions includes various splicing abnormal events and events affecting images caused by humans at different resolutions, as well as event effect ratings;

[0080] Assigning preset impact assessment values ​​to various impact event types of image splicing at different resolutions in the impact event set of image splicing at different resolutions based on event effect ratings;

[0081] A regression model is constructed based on the set of impact events of image splicing at different resolutions and the corresponding impact assessment values ​​to generate the image splicing event impact assessment coefficient.

[0082] The specific calculation formula of the image splicing event impact assessment coefficient is as follows:

[0083]

[0084] In the formula, is the image stitching event impact assessment coefficient, … The preset impact assessment values ​​corresponding to the impact events of image splicing at different resolutions, … is the proportion of the influencing events of image splicing at different resolutions to the influencing events of image splicing at all different resolutions, is the type of influencing event of image splicing at different resolutions, n is the number of influencing event of image splicing at different resolutions, and e is a natural number.

[0085] This embodiment achieves comprehensive optimization of real-time video panoramic stitching performance under different input resolutions by integrating modules such as data acquisition, comprehensive evaluation, input data volume judgment, discrimination constraints, and resolution input and output. The system can accurately quantify the impact of response rate, image quality, and stitching events, and then select the optimal input resolution that meets system performance and user experience requirements. The response rate evaluation coefficient effectively reduces system latency and improves real-time performance; the image quality impact coefficient ensures image clarity and visual effects; the image stitching event impact evaluation coefficient reduces visual defects during the stitching process. Combining these evaluation coefficients, the system can scientifically and rationally optimize resolution selection, improve operational accuracy and safety, and enhance user experience.

[0086] Embodiment 2, in this embodiment, the comprehensive evaluation module is used to construct a resolution impact evaluation model based on long-term and short-term neural networks according to system response data and image sensory data.

[0087] The resolution impact assessment model is constructed based on the long-term and short-term neural network according to the system response data and the image sensory data, specifically:

[0088] The obtained response rate assessment coefficient, image quality impact coefficient and image stitching event impact assessment coefficient are used to establish a resolution impact assessment model to generate a resolution impact assessment coefficient;

[0089] The specific calculation formula of the resolution impact assessment coefficient is as follows:

[0090]

[0091] In the formula, is the resolution impact assessment coefficient, is the preset scaling factor for the response rate evaluation coefficient, is the preset proportional coefficient of the image quality impact coefficient, The preset proportional coefficient for the image stitching event impact assessment coefficient, is the response rate evaluation coefficient, is the image quality influence coefficient, is the impact assessment coefficient of the image stitching event.

[0092] In this embodiment, the input data volume determination module is used to construct an input resolution-stitching effect curve diagram according to the output of the resolution impact assessment model under different input resolutions.

[0093] According to the output of the resolution impact assessment model under different input resolutions, an input resolution-splicing effect curve is constructed, which is specifically:

[0094] Obtaining resolution impact assessment coefficients of outputs of input resolutions under different settings from a resolution impact assessment model under different input resolutions;

[0095] A rectangular coordinate system is established, with the horizontal axis representing the input resolution and the vertical axis representing the resolution impact assessment coefficient;

[0096] Mark the resolution impact assessment coefficient corresponding to each input resolution on the rectangular coordinate system to form a data point;

[0097] Connect the data points to form a curve graph of input resolution-stitching effect.

[0098] In this embodiment, the discrimination constraint module is used to perform data analysis and construct discrimination constraint conditions based on hardware data and manually preset restrictions.

[0099] The method for obtaining the discrimination constraint condition is as follows:

[0100] Collecting system hardware performance data, including processing power, storage limitations and bandwidth, and manually set image quality requirements, response time and stability standards;

[0101] By establishing performance models and constraint models, performance constraints and quality standards are integrated to construct discrimination constraints.

[0102] In this embodiment, the resolution input and output module is used to cut the input resolution-stitching effect curve graph according to the discrimination constraint condition, and take the peak value of the cut resolution-stitching effect curve graph as the optimal input resolution, and perform panoramic stitching.

[0103] The input resolution-splicing effect curve graph is cut according to the discrimination constraint condition, specifically:

[0104] The curves in the input resolution-stitching effect curve graph that do not meet the discrimination constraint conditions are removed, and the input resolution-stitching effect curve graph that meets the discrimination constraint conditions is left.

[0105] This implementation uses a comprehensive evaluation module, an input data volume judgment module, a discrimination constraint module, and a resolution input / output module, and uses a long-term and short-term neural network to build a resolution impact evaluation model to optimize the input resolution selection of the tower crane control system; this embodiment integrates system response data, image quality impact data, and splicing event impact data into a resolution impact evaluation coefficient, and based on this, establishes an input resolution-splicing effect curve, effectively transforming performance and quality standards into an operational decision-making basis. Furthermore, the discrimination constraint module combines hardware performance and preset standards to screen the curve to ensure that only resolutions that meet the requirements are retained, thereby optimizing the panoramic splicing effect and improving system performance and user experience.

[0106] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0107] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0108] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0110] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0111] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The tower crane control system based on real-time video panoramic stitching technology is characterized by: It includes a data acquisition module, a comprehensive evaluation module, an input data volume judgment module, a judgment constraint module, and a resolution input and output module; A data acquisition module, used to acquire system response data and image data of real-time video panoramic stitching at different input resolutions; The system response data includes a response rate evaluation coefficient; the image data includes an image quality impact coefficient and an image splicing event impact evaluation coefficient; A comprehensive evaluation module for building a resolution impact evaluation model based on long-term and short-term neural networks according to system response data and image sensory data; The resolution impact assessment model is constructed based on the long-term and short-term neural network according to the system response data and the image sensory data, specifically: The acquired response rate assessment coefficient, image quality impact coefficient and image stitching event impact assessment coefficient are used to establish a resolution impact assessment model to generate a resolution impact assessment coefficient; The specific calculation formula of the resolution impact assessment coefficient is as follows: In the formula, is the resolution impact assessment coefficient, is the preset scaling factor for the response rate evaluation coefficient, is the preset proportional coefficient of the image quality impact coefficient, The preset proportional coefficient for the image stitching event impact assessment coefficient, is the response rate evaluation coefficient, is the image quality influence coefficient, is the impact assessment coefficient of image splicing events; The input data volume judgment module is used to construct an input resolution-stitching effect curve diagram according to the output of the resolution impact assessment model under different input resolutions; The discrimination constraint module is used to perform data analysis and construct discrimination constraint conditions based on hardware data and manually preset restrictions; The resolution input and output module is used to cut the input resolution-stitching effect curve according to the judgment constraint conditions, and use the peak value of the cut resolution-stitching effect curve as the optimal input resolution, and perform panoramic stitching.

2. The tower crane control system based on real-time video panoramic stitching technology according to claim 1 is characterized in that: The specific method for obtaining the response rate evaluation coefficient is as follows: Under different input resolutions, the time required for the system to receive a video frame, complete processing, and generate a stitching result is obtained as the delay time; For the same resolution, multiple measurements are performed, abnormal delay times are removed, and the average delay time is calculated; Obtain the number of frames processed by the system at different input resolutions and the total time required for processing, and calculate the system video processing rate; An initial experience diminishing effect model is set based on a power function model according to the average delay time and system video processing rate; Set several delay time points to conduct user experience testing, and collect user experience rating data at each delay time point; According to the experience rating data, the undetermined parameters of the initial experience diminishing effect model are optimized based on the nonlinear least squares method to obtain a calculation model for the response rate evaluation coefficient. The undetermined parameters include the maximum value of the user experience, the rate of decrease of the experience with increasing delay, and the sensitivity coefficient of delay to the diminishing experience.

3. The tower crane control system based on real-time video panoramic stitching technology according to claim 2 is characterized in that: The specific method for obtaining the image quality influence coefficient is as follows: The positive stitching deformation generated by panoramic stitching through the panoramic image intake points sets the positive stitching deformation reference value, and the negative stitching deformation sets the negative stitching deformation reference value. The positive stitching deformation less than the positive stitching deformation reference value is set as an acceptable positive stitching deformation, and the positive stitching deformation greater than the positive stitching deformation reference value is set as a potential positive stitching deformation; the negative stitching deformation less than the negative stitching deformation reference value is set as a potential negative stitching deformation, and the negative stitching deformation greater than the negative stitching deformation reference value is set as an acceptable negative stitching deformation; Acquire positive stitching deformation values ​​and negative stitching deformation values ​​of the panoramic image intake point, compare the positive stitching deformation value with the positive stitching deformation reference value, establish a positive stitching deformation set with the positive stitching deformation values ​​of all hidden positive stitching deformations, and establish a negative stitching deformation set with the negative stitching deformation values ​​of all hidden negative stitching deformations; Data analysis is performed based on the positive splicing deformation set and the negative splicing deformation set, and the image quality impact coefficient is calculated.

4. The tower crane control system based on real-time video panoramic stitching technology according to claim 3 is characterized in that: The specific method for obtaining the image splicing event impact assessment coefficient is as follows: Based on the historical event types of image splicing at different resolutions, a set of events affecting image splicing at different resolutions is obtained, wherein the set of events affecting image splicing at different resolutions includes various splicing abnormal events and events affecting images caused by humans at different resolutions, as well as event effect ratings; Assigning preset impact assessment values ​​to various impact event types of image splicing at different resolutions in the impact event set of image splicing at different resolutions based on event effect ratings; A regression model is constructed based on the set of impact events of image splicing at different resolutions and the corresponding impact assessment values ​​to generate the image splicing event impact assessment coefficient.

5. The tower crane control system based on real-time video panoramic stitching technology according to claim 4 is characterized in that: According to the output of the resolution impact assessment model under different input resolutions, an input resolution-splicing effect curve is constructed, which is specifically: Obtaining resolution impact assessment coefficients of outputs of input resolutions under different settings from a resolution impact assessment model under different input resolutions; A rectangular coordinate system is established, with the horizontal axis representing the input resolution and the vertical axis representing the resolution impact assessment coefficient; Mark the resolution impact assessment coefficient corresponding to each input resolution on the rectangular coordinate system to form a data point; Connect the data points to form a curve graph of input resolution-stitching effect.

6. The tower crane control system based on real-time video panoramic stitching technology according to claim 5 is characterized in that: The method for obtaining the discrimination constraint condition is as follows: Collecting system hardware performance data, including processing power, storage limitations and bandwidth, and manually set image quality requirements, response time and stability standards; By establishing performance models and constraint models, performance constraints and quality standards are integrated to construct discrimination constraints.

7. The tower crane control system based on real-time video panoramic stitching technology according to claim 6 is characterized in that: The input resolution-splicing effect curve graph is cut according to the discrimination constraint condition, specifically: The curves in the input resolution-stitching effect curve graph that do not meet the discrimination constraint conditions are removed, and the input resolution-stitching effect curve graph that meets the discrimination constraint conditions is left.

8. The tower crane control system based on real-time video panoramic stitching technology according to claim 7 is characterized in that: The specific calculation formula of the average delay time is as follows: The specific calculation formula of the error function included in the nonlinear least squares method is as follows: The specific calculation formula of the response rate evaluation coefficient is as follows: In the formula, is the average delay time, is the delay time of the ith measurement, To measure the delay times at the same input resolution, is the error function included in the nonlinear least squares method, The amount of user experience rating data used for optimization, The label for the user's experience rating data. Provides a user experience rating data value. is the response rate evaluation coefficient.

9. The tower crane control system based on real-time video panoramic stitching technology according to claim 8 is characterized in that: The specific calculation formula of the image quality influence coefficient is as follows: In the formula, is the image quality influence coefficient, is the positive splicing deformation value in the positive splicing deformation set, h is the number of the positive splicing deformation value in the positive splicing deformation set, is the negative splicing deformation value in the negative splicing deformation set, k is the number of the positive splicing deformation value in the positive splicing deformation set, , The time period for obtaining the forward stitching deformation value of the panoramic image intake point within the P time interval and the Q time interval, , The time period for obtaining negative stitching deformation values ​​of the panoramic image intake point within the P time and Q time interval.

Images