Adaptive visual compensation system and method

Abstract

This application relates to the technical field of visual compensation and discloses an adaptive visual compensation system and method. The system includes a video analysis module, an environmental perception module, a scene recognition module, a prediction module, a compensation strategy module, and an algorithm optimization module. The system comprises: a video analysis module for analyzing a real-time input video stream and extracting feature data from the video stream; an environmental perception module for collecting environmental parameters; a scene recognition module for determining the scene type; a prediction module for predicting the changing trend of the environmental parameters and the user's optimization goals; a compensation strategy module for determining the visual compensation strategy for the video stream; and an algorithm optimization module for adaptively selecting a visual compensation algorithm and performing visual compensation on the video stream. This application implements adaptive visual compensation for multiple scenes, significantly improving the real-time and accuracy of video stream processing in complex environments.

Description

Technical Field

[0001] The present application relates to the technical field of vision compensation, and in particular to an adaptive vision compensation system and method. Background Art

[0002] Current vision compensation technologies primarily rely on two approaches: fixed-parameter compensation and offline training models. Fixed-parameter compensation processes images based on preset rules (such as illumination thresholds and fixed filter kernels). Typical applications include gamma correction for industrial cameras and electronic image stabilization for security cameras. While computationally efficient, these methods struggle to adapt to dynamic environmental changes. Offline training models, which use deep learning (such as CNNs and GANs) to train specialized compensation models, have achieved some success in fields such as medical endoscopy and autonomous driving. However, the model's generalization ability is limited by the training data and consumes significant computational resources.

[0003] Current mainstream visual compensation technologies in the industry suffer from significant flaws. Industrial, medical, and security sectors employ independent technology approaches, lacking a unified architecture, resulting in fragmented scenarios and high cross-domain migration costs. When the system cannot identify the scene type, it employs a generic compensation model, ignoring the differences between environmental characteristics and user needs. Compensation parameters in existing visual compensation systems are typically set empirically and cannot be adjusted in real time to environmental changes. Existing systems also fail to incorporate user intent analysis, leading to a mismatch between compensation results and actual needs.

[0004] For example, patent application publication number CN112534467A discloses a system and method for contrast sensitivity compensation, designed to correct the vision of users whose vision is insufficient to discern high spatial frequencies. This technical solution can use the user's contrast detection as a function of the spatial frequency in an image to correct the image in real time. However, the problem raised in the background of this application still exists: the compensation strategy cannot be adjusted in real time based on environmental changes.

[0005] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the application and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the Invention

[0006] The technical problem to be solved by this application is to overcome the defects of the existing technology and provide an adaptive visual compensation system and method to realize adaptive visual compensation in multiple scenes and improve the real-time and accuracy of video stream processing in complex environments.

[0007] To solve the above technical problems, this application provides the following technical solutions:

[0008] On the one hand, the present application provides an adaptive visual compensation system, including a video analysis module, an environment perception module, a scene recognition module, a prediction module, a compensation strategy module, and an algorithm optimization module; wherein:

[0009] The video analysis module is used to analyze the real-time input video stream and extract the feature data of the video stream;

[0010] The environment perception module is used to collect the environmental parameters of the environment corresponding to the video stream;

[0011] The scene recognition module performs scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type;

[0012] The prediction module is used to predict the change trend of the environmental parameters; the prediction module is also used to predict the user's optimization target;

[0013] The compensation strategy module determines a visual compensation strategy for the video stream based on the scene type, the changing trend of the environmental parameters, and the user's optimization goal;

[0014] The algorithm optimization module adaptively selects a visual compensation algorithm based on the visual compensation strategy, and performs visual compensation on the video stream based on the visual compensation algorithm.

[0015] As a preferred solution of the adaptive visual compensation system described in the present application, wherein: the characteristic data of the video stream includes resolution, frame rate, encoding format, degree of geometric distortion, and color offset;

[0016] The video analysis module includes an information extraction unit, a distortion analysis unit, and a color analysis unit; wherein the information extraction unit is used to extract the resolution, frame rate, and encoding format of the video stream;

[0017] The distortion analysis unit is used to extract the degree of geometric distortion of the video stream;

[0018] The color analysis unit is used to extract the color offset of the video stream;

[0019] The environmental parameters include ambient light intensity, lens acceleration, and operation trajectory data;

[0020] The environment perception module includes a sensor unit and an operation recording unit; wherein the sensor unit is used to collect ambient light intensity and lens acceleration;

[0021] The operation recording unit is used to obtain the user's operation trajectory data; the operation trajectory data includes the screen coordinates of each touch operation of the user in the last m seconds, where m is a positive integer.

[0022] As a preferred solution of the adaptive visual compensation system described in this application, wherein: the scene types include industrial detection scenes, medical imaging scenes, and security monitoring scenes;

[0023] The scene recognition module includes a scene classification unit and a scene verification unit; wherein the scene classification unit calculates the confidence of each scene type based on the video frame;

[0024] The scene verification unit determines the scene type by performing cross-verification based on the confidence of each scene type and the feature data of the video stream.

[0025] As a preferred solution of the adaptive visual compensation system described in the present application, wherein: the prediction module includes an environment prediction unit and a target prediction unit;

[0026] The environmental prediction unit is used to predict the change trend of the environmental parameters, specifically including:

[0027] Extract Past Continuous The ambient light intensity and lens acceleration corresponding to the frame video are normalized respectively; the lens acceleration is decomposed into the three-axis acceleration in the three-dimensional coordinate system; the past continuous The ambient light intensity corresponding to the frame video and the three-axis acceleration of the lens are input into the trained LSTM network model to predict and output the future continuous The ambient light intensity corresponding to the frame video and the three-axis acceleration of the lens; 、 are all positive integers;

[0028] Based on future continuous The ambient light intensity corresponding to the frame video is used to calculate the light intensity gradient;

[0029] Based on future continuous The three-axis acceleration of the lens corresponding to the frame video is used to determine whether the lens has undergone sudden motion changes;

[0030] The target prediction unit is used to predict the user's optimization target, specifically including: extracting the screen coordinates of each touch operation of the user in the last m seconds based on the operation trajectory data, and generating the touch operation probability of each point in the video;

[0031] Detecting a motion area in a video stream; and identifying a user's potential focus area by combining the touch operation probability with the motion area.

[0032] As a preferred solution of the adaptive visual compensation system described in the present application, the compensation strategy module includes a first strategy unit; the first strategy unit is used to determine the visual compensation strategy of the video stream, specifically including:

[0033] The first strategy unit is configured with a light intensity threshold and an acceleration threshold; if the ambient light intensity is lower than the light intensity threshold, the visual compensation strategy of the video stream includes a brightness compensation algorithm;

[0034] If the lens acceleration is greater than the acceleration threshold, the visual compensation strategy of the video stream includes a blur correction algorithm;

[0035] If a potential area of interest of the user is identified, the visual compensation strategy of the video stream further includes a local enhancement algorithm for locally enhancing the potential area of interest.

[0036] As a preferred solution of the adaptive visual compensation system described in the present application, the first strategy unit is further configured with a first distortion threshold and a second distortion threshold; if the current scene type is an industrial inspection scene, when the degree of geometric distortion is greater than the first distortion threshold, the visual compensation strategy of the video stream also includes a distortion compensation algorithm; otherwise, when the degree of geometric distortion is greater than the second distortion threshold, the visual compensation strategy of the video stream also includes a distortion compensation algorithm; the first distortion threshold is greater than the second distortion threshold;

[0037] The first strategy unit is also configured with a first color difference threshold and a second color difference threshold; if the current scene type is a medical imaging scene, when the color offset is greater than the first color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm; otherwise, when the color offset is greater than the second color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm; the first color difference threshold is less than the second color difference threshold.

[0038] As a preferred solution of the adaptive visual compensation system described in the present application, the compensation strategy module further includes a second strategy unit; the second strategy unit optimizes and adjusts the visual compensation strategy based on the changing trend of the environmental parameters, specifically including:

[0039] The second strategy unit is configured with a gradient threshold of light intensity; if the light intensity gradient is greater than the gradient threshold, and the visual compensation strategy includes a brightness compensation algorithm, increasing the iteration frequency of the brightness compensation algorithm;

[0040] If the lens motion changes suddenly and the visual compensation strategy includes a blur correction algorithm, a multi-frame buffer mechanism is added to the blur suppression algorithm.

[0041] As a preferred solution of the adaptive vision compensation system described in the present application, the algorithm optimization module includes an algorithm optimization unit; the algorithm optimization unit adaptively selects a vision compensation algorithm based on the vision compensation strategy, specifically including:

[0042] Obtain an alternative algorithm for each algorithm included in the vision compensation strategy;

[0043] Generating an algorithm combination of a visual compensation strategy; any group of the algorithm combinations includes an alternative algorithm for each algorithm included in the visual compensation strategy;

[0044] Set the optimization objective function based on the scenario type;

[0045] Calculate the optimization target value of each algorithm combination based on the optimization objective function;

[0046] The algorithm combination with the largest optimization target value is selected and transmitted to the visual compensation unit.

[0047] As a preferred solution of the adaptive visual compensation system described in the present application, the algorithm optimization module further includes a visual compensation unit; the visual compensation unit performs visual compensation on the video stream based on the visual compensation algorithm, specifically including:

[0048] Preprocessing the input video stream; the preprocessing includes frame alignment and noise suppression;

[0049] The visual compensation unit is configured with an alternative algorithm for each algorithm included in the visual compensation strategy; based on the algorithm combination selected by the algorithm optimization unit, the video stream is visually compensated and a video stream with completed visual compensation is output.

[0050] In a second aspect, the present application provides an adaptive visual compensation method, comprising the following steps:

[0051] Analyze the real-time input video stream and extract the characteristic data of the video stream; collect the environmental parameters of the environment corresponding to the video stream;

[0052] Performing scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type;

[0053] Predicting the changing trends of the environmental parameters and the user's optimization goals;

[0054] Determining a visual compensation strategy for the video stream based on the scene type, the changing trend of the environmental parameters, and the user's optimization goal;

[0055] A visual compensation algorithm is adaptively selected based on the visual compensation strategy, and visual compensation is performed on the video stream based on the visual compensation algorithm.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The vision compensation system provided in this application has multi-scenario adaptive capabilities. Through a unified system architecture, it adapts to the differentiated needs of different scenarios such as industry, medical care, and security, avoiding the redundant design of one solution for each scenario in traditional technologies and achieving cross-domain compatibility.

[0058] This application uses the LSTM network to predict the light intensity gradient and motion mutation trend, and adjusts the visual compensation algorithm parameters in advance to solve the response lag problem of traditional methods.

[0059] By analyzing user operation trajectories and screen content features, it can identify potential areas of user attention, enhancing the practical value of the compensation results. Dynamically configuring optimization function weights based on scene types achieves a precise balance between computing resources and visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them:

[0061] Figure 1 A schematic diagram of the structure of an adaptive visual compensation system provided by this application;

[0062] Figure 2 This is a flowchart of an adaptive visual compensation method provided by this application. DETAILED DESCRIPTION

[0063] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0064] Example 1

[0065] This embodiment introduces an adaptive visual compensation system. Figure 1 The system includes a video analysis module, an environmental perception module, a scene recognition module, a prediction module, a compensation strategy module, and an algorithm optimization module; wherein:

[0066] The video analysis module is used to analyze the real-time input video stream and extract the feature data of the video stream;

[0067] The characteristic data of the video stream include resolution, frame rate, encoding format, degree of geometric distortion, and color offset;

[0068] The video analysis module includes an information extraction unit, a distortion analysis unit, and a color analysis unit; the information extraction unit is used to extract the resolution, frame rate, and encoding format of the video stream; the information extraction unit parses the metadata of the video stream or analyzes the video frames to obtain the width and height pixel values of the video, and then determines the resolution; the frame rate is determined by calculating the time interval between adjacent video frames; and the file header or metadata of the video stream is parsed to identify the encoding format used, such as H.264, H.265, etc.

[0069] The distortion analysis unit is used to extract the degree of geometric distortion of the video stream; the distortion analysis unit finds feature points in the video frame, including corner points, edge points, etc.; uses the feature points to fit a geometric distortion model, including but not limited to perspective distortion, radial distortion, etc.; and calculates the degree of geometric distortion based on the fitted distortion model.

[0070] The color analysis unit is used to extract the color offset of the video stream. The color analysis unit extracts the color features of the video frame, including the mean and variance; compares the extracted color features with the preset reference color features, and calculates the color offset.

[0071] The environment perception module is used to collect the environmental parameters of the environment corresponding to the video stream;

[0072] The environmental parameters include ambient light intensity, lens acceleration, and operation trajectory data;

[0073] The environment perception module includes a sensor unit and an operation recording unit; wherein the sensor unit is used to collect ambient light intensity and lens acceleration;

[0074] The operation recording unit is configured to obtain user operation trajectory data; the operation trajectory data includes the screen coordinates of each user touch operation within the last m seconds. m is a positive integer. For example, if m is 30, the operation trajectory data is constructed based on the screen coordinates of the user's touch operations within the last 30 seconds.

[0075] The scene recognition module performs scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type;

[0076] The scene types include industrial detection scenes, medical imaging scenes, and security monitoring scenes;

[0077] The scene recognition module includes a scene classification unit and a scene verification unit; wherein, the scene classification unit calculates the confidence of each scene type based on the video frame; the scene classification unit is configured with a pre-trained ResNet-50 scene classifier, which is used to perform scene recognition based on the video frame and output the confidence that the current scene is each scene type.

[0078] The scene verification unit determines the scene type by cross-verifying the confidence level of each scene type with the feature data of the video stream. In this embodiment, the preferred method for determining the scene type includes:

[0079] If the confidence level of a medical imaging scene is ≥0.8, the resolution of the video stream feature data is ≥3840×2160 and the encoding format complies with the DICOM standard, and the synthetic vector amplitude of the lens acceleration is <0.5g (g is the acceleration due to gravity), then it is determined to be a medical imaging scene.

[0080] If the confidence level of an industrial inspection scene is ≥0.7, the frame rate of the video stream is ≥60 fps, and there are periodic brightness fluctuations, for example, a brightness peak occurs every 0.5±0.1 seconds, and the user has no touch operation for 10 consecutive seconds, then the scene is determined to be an industrial inspection scene.

[0081] If the confidence level of the security monitoring scene is ≥0.6 and there are multiple moving targets in the video stream, for example, ≥3 motion vector areas are detected by the optical flow method, and the encoding parameters are H.264 and the GOP length is >10 frames, then it is determined to be a security monitoring scene.

[0082] Furthermore, if it cannot be confirmed that the current scene is any of the industrial inspection scenes, medical imaging scenes, and security monitoring scenes, the scene type will be marked as unknown, and basic adaptive visual compensation will be performed on the video stream; the preferred basic adaptive visual compensation in this embodiment specifically includes: dynamically adjusting the global brightness and contrast according to the ambient light intensity, enabling nonlinear stretching in low light, and starting highlight suppression in overexposure; judging the degree of jitter based on lens acceleration data, adaptively selecting the anti-shake algorithm, using electronic image stabilization for light jitter, and enabling multi-frame fusion for strong jitter; analyzing the high-frequency interaction area in the operation trajectory data, and giving priority to local sharpening and noise reduction in this area; matching the basic enhancement algorithm according to the video stream resolution and frame rate, disabling super-resolution processing at low resolution, and enhancing time domain noise reduction at high frame rate.

[0083] The prediction module is used to predict the change trend of the environmental parameters; the prediction module is also used to predict the user's optimization target;

[0084] The prediction module includes an environment prediction unit and a target prediction unit;

[0085] The environmental prediction unit is used to predict the change trend of the environmental parameters, specifically including:

[0086] Extract Past Continuous The ambient light intensity and lens acceleration corresponding to the frame video are normalized respectively; the lens acceleration is decomposed into the three-axis acceleration in the three-dimensional coordinate system; the past continuous The ambient light intensity corresponding to the frame video and the three-axis acceleration of the lens are input into the trained LSTM network model to predict and output the future continuous The ambient light intensity corresponding to the frame video and the three-axis acceleration of the lens; 、 are all positive integers; The value is 32. The value is 8. The changes in ambient light intensity and the three-axis acceleration of the lens are continuous. This embodiment uses a longer 32-frame data to ensure that the model input contains sufficient information. The predictive ability of the LSTM network model gradually decreases with increasing output step size. Therefore, this embodiment outputs a shorter 8-frame data while meeting the requirement for predicting the changing trend of environmental parameters.

[0087] Based on future continuous The ambient light intensity corresponding to the frame video is used to calculate the light intensity gradient;

[0088] Based on future continuous The three-axis acceleration of the lens corresponding to the frame video is used to determine whether the lens has a sudden motion change. For example, calculate the future continuous The composite acceleration of the three-axis acceleration corresponding to each frame in the frame; if the composite acceleration of any frame exceeds the preset acceleration threshold, it is determined that the lens motion mutation occurs.

[0089] The target prediction unit is used to predict the user's optimization target, specifically including: extracting the screen coordinates of each touch operation of the user in the last m seconds based on the operation trajectory data, and generating the touch operation probability of each point in the video;

[0090] Detecting a motion area in a video stream; and identifying a user's potential focus area by combining the touch operation probability with the motion area.

[0091] The method for the target prediction unit preferred in the present application to identify the user's potential focus area is as follows: Gaussian kernel convolution is used to convert the screen coordinates of discrete touch operations into a continuous probability density distribution of touch probabilities, and a touch probability density map is constructed; the motion area is detected from the video stream through a lightweight optical flow method; the touch probability of the coordinate point in the probability density map corresponding to the motion area is increased by 30%; a touch probability threshold is set, and all coordinate points with a touch probability greater than the touch probability threshold are marked as points of interest; and adaptive threshold segmentation is used to generate a connected domain of points of interest as the user's potential focus area.

[0092] The compensation strategy module determines a visual compensation strategy for the video stream based on the scene type, the changing trend of the environmental parameters, and the user's optimization goal;

[0093] The compensation strategy module includes a first strategy unit and a second strategy unit;

[0094] The first strategy unit is used to determine a visual compensation strategy for a video stream, specifically including:

[0095] The first strategy unit is configured with a light intensity threshold and an acceleration threshold; if the ambient light intensity is lower than the light intensity threshold, the visual compensation strategy of the video stream includes a brightness compensation algorithm; in this embodiment, the preferred light intensity threshold is 50 lux; the brightness compensation algorithm includes a histogram equalization algorithm, a Retinex algorithm, etc., which are used to improve the image brightness.

[0096] If the lens acceleration is greater than the acceleration threshold, the visual compensation strategy of the video stream includes a blur correction algorithm; in this embodiment, 2 times the acceleration of gravity is preferably used as the acceleration threshold; the blur correction algorithm includes a Wiener filter algorithm, a Laplace sharpening algorithm, etc., which is used to perform blur correction on the video stream;

[0097] If the user's potential area of interest is identified, the visual compensation strategy of the video stream also includes a local enhancement algorithm for locally enhancing the potential area of interest; for example, superimposing a Gaussian mask in the spatial domain of the video stream to enhance the details of the target area.

[0098] The first strategy unit is also configured with a first distortion threshold and a second distortion threshold; if the current scene type is an industrial inspection scene, then when the degree of geometric distortion is greater than the first distortion threshold, the visual compensation strategy of the video stream also includes a distortion compensation algorithm; otherwise, when the degree of geometric distortion is greater than the second distortion threshold, the visual compensation strategy of the video stream also includes a distortion compensation algorithm; the first distortion threshold is greater than the second distortion threshold. Industrial inspection scenes pay more attention to the geometric size and position accuracy of the inspection target, such as the measurement of the gap between parts assembly, rather than the absolute deformation of the edge of the picture. Moderately retaining distortion can exchange for a larger field of view coverage and improve inspection efficiency. Distortion compensation algorithms include polynomial correction method, bilinear interpolation method, etc., which are used to perform distortion compensation correction on video streams.

[0099] The first strategy unit is also configured with a first color difference threshold and a second color difference threshold. If the current scene type is a medical imaging scene, then when the color offset is greater than the first color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm. Otherwise, when the color offset is greater than the second color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm. The first color difference threshold is less than the second color difference threshold. Medical imaging scenes have high requirements for color discrimination, and a more sensitive color difference threshold is set to trigger color correction. The color correction algorithm includes color space conversion, grayscale world assumption, etc., which are used to perform color correction on the video stream.

[0100] The second strategy unit optimizes and adjusts the visual compensation strategy based on the change trend of the environmental parameter, specifically including:

[0101] The second strategy unit is configured with a gradient threshold of light intensity; if the light intensity gradient is greater than the gradient threshold and the visual compensation strategy includes a brightness compensation algorithm, the iteration frequency of the brightness compensation algorithm is increased; for example, the basic iteration frequency of the brightness compensation algorithm is 30Hz, which is increased to 60Hz to match the changing speed of the ambient light.

[0102] If the lens experiences sudden motion changes and the visual compensation strategy includes a blur correction algorithm, a multi-frame buffer mechanism is added to the blur suppression algorithm. This multi-frame buffer mechanism ensures that the blur suppression error compensation rate is reduced when there is a sudden change in motion.

[0103] The algorithm optimization module adaptively selects a visual compensation algorithm based on the visual compensation strategy, and performs visual compensation on the video stream based on the visual compensation algorithm.

[0104] The algorithm optimization module includes an algorithm optimization unit and a vision compensation unit;

[0105] The algorithm optimization unit adaptively selects a vision compensation algorithm based on the vision compensation strategy, specifically including:

[0106] Obtain alternative algorithms for each algorithm included in the visual compensation strategy; for example, the visual compensation strategy includes a distortion compensation algorithm, which describes a type of visual compensation algorithm. The specific alternative algorithms corresponding to this type include polynomial correction method, bilinear interpolation method, perspective transformation correction method, etc. These alternative algorithms can all achieve distortion compensation for the video stream.

[0107] Generating an algorithm combination of a visual compensation strategy; any group of the algorithm combinations includes an alternative algorithm for each algorithm included in the visual compensation strategy;

[0108] Set the optimization objective function based on the scenario type;

[0109] Calculate the optimization target value of each algorithm combination based on the optimization objective function;

[0110] The algorithm combination with the largest optimization target value is selected and transmitted to the visual compensation unit.

[0111] In this embodiment, the optimization objective function is preferably set based on the scenario type, and the method of calculating the optimization objective value of each algorithm combination includes but is not limited to:

[0112] For each candidate algorithm included in the visual compensation strategy, a real-time score, a resolution score, a motion clarity score, and a low-light performance score are assigned. The real-time score is the ratio of the algorithm's processing delay to the frame period; the resolution score is the degree of resolution retention of the algorithm's output image; the motion clarity score is the inverse of the proportion of artifacts in moving areas after processing by the algorithm; and the low-light performance score is calculated based on the color difference after processing by the algorithm. For example, the Retinex-based adaptive color correction algorithm, due to its characteristics such as requiring iterative calculation of illumination components, maintaining original resolution, excellent chromatic noise suppression capabilities, and the potential for halo artifacts in moving areas, has a higher resolution score and low-light performance score calculated according to the above rules, but a lower real-time score and motion clarity score.

[0113] The real-time score, resolution score, motion clarity score, and low-light performance score of each candidate algorithm are weighted and summed to obtain a comprehensive score for the candidate algorithm. The weight values for the weighted summation are set based on the scenario type. For example, for industrial inspection scenarios, industrial inspection is usually strongly related to conveyor belt speed, and the algorithm delay must be much lower than the production line cycle time. Therefore, the real-time score has a higher weight, 0.8. Industrial parts usually have fixed geometric features and do not require ultra-high resolution. Therefore, the resolution score has a lower weight, 0.2. Conveyor belt vibration / robotic arm movement may cause dynamic blur, so the motion clarity score has a higher weight, 0.5. Industrial environment lighting is relatively controllable and low-light conditions are rare. Therefore, the low-light performance score has a lower weight, 0.2. Those skilled in the art can refer to the ideas provided in this embodiment and set the optimization objective function for medical imaging scenarios and security monitoring scenarios according to actual needs. No further details will be given.

[0114] Calculate the sum of the comprehensive scores of each alternative algorithm in each algorithm combination to obtain the optimization target value of the algorithm combination.

[0115] The visual compensation unit performs visual compensation on the video stream based on a visual compensation algorithm, specifically including:

[0116] Preprocessing the input video stream; in this embodiment, the preprocessing preferably includes frame alignment and noise suppression;

[0117] The visual compensation unit is configured with an alternative algorithm for each algorithm included in the visual compensation strategy; based on the algorithm combination selected by the algorithm optimization unit, the video stream is visually compensated and a video stream with completed visual compensation is output.

[0118] Example 2

[0119] This embodiment is the second embodiment of the present application; it is based on the same inventive concept as embodiment 1, and Figure 2This embodiment introduces an adaptive visual compensation method, including the following steps:

[0120] Analyze the real-time input video stream and extract its feature data; collect the environmental parameters of the video stream; the feature data includes resolution, frame rate, encoding format, geometric distortion, and color offset; the environmental parameters include ambient light intensity, lens acceleration, and operation trajectory data;

[0121] Perform scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type; the scene types include industrial detection scenes, medical imaging scenes, and security monitoring scenes;

[0122] Predicting the changing trend of the environmental parameters and the user's optimization target; the changing trend of the environmental parameters includes the illumination intensity gradient and whether the lens has a sudden change in motion; the user's optimization target includes the user's potential focus area.

[0123] Determine the visual compensation strategy of the video stream based on the scene type, the changing trend of environmental parameters, and the user's optimization goal; the visual compensation strategy includes brightness compensation algorithm, blur correction algorithm, local enhancement algorithm, distortion compensation algorithm, color correction algorithm, etc.;

[0124] Based on the visual compensation strategy, a visual compensation algorithm is adaptively selected, and visual compensation is performed on the video stream based on the visual compensation algorithm. Different optimization objective functions are set based on the scene type, and the best algorithm combination is selected to perform visual compensation.

[0125] The specific functional implementation of each of the above steps refers to the relevant content of the adaptive visual compensation system described in Example 1 and is not described in detail here.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] The above describes the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of this application, all of which are protected by this application.

Claims

1. An adaptive visual compensation system, characterized by: It includes video analysis module, environment perception module, scene recognition module, prediction module, compensation strategy module, and algorithm optimization module; among them: The video analysis module is used to analyze the real-time input video stream and extract the feature data of the video stream; The environment perception module is used to collect the environmental parameters of the environment corresponding to the video stream; The scene recognition module performs scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type; The prediction module is used to predict the change trend of the environmental parameters and predict the user's optimization goals; The compensation strategy module determines a visual compensation strategy for the video stream based on the scene type, the changing trend of the environmental parameters, and the user's optimization goal; the visual compensation strategy is a collection of types of visual compensation algorithms; The algorithm optimization module adaptively selects a visual compensation algorithm based on the visual compensation strategy, specifically including: setting a corresponding alternative algorithm for each visual compensation algorithm in the visual compensation strategy; determining which of the corresponding alternative algorithms each visual compensation algorithm in the visual compensation strategy is based on the scene type; Each adaptively selected visual compensation algorithm is used in turn to process the video stream; The prediction module includes an environment prediction unit and a target prediction unit; The environment prediction unit is used to predict the change trend of the environmental parameters, specifically including: extracting the past continuous The ambient light intensity and lens acceleration corresponding to the frame video are processed and input into the trained LSTM network model to predict and output the future continuous The ambient light intensity corresponding to the frame video and the three-axis acceleration of the lens; 、 are all positive integers; Based on future continuous The ambient light intensity corresponding to the frame video is used to calculate the light intensity gradient; Based on future continuous The three-axis acceleration of the lens corresponding to the frame video is used to determine whether the lens has undergone sudden motion changes; The target prediction unit is used to predict the user's optimization target. Specifically, it extracts the screen coordinates of each touch operation of the user in the last m seconds based on the operation trajectory data, and generates the touch operation probability of each point in the video. Detecting motion areas in the video stream; combining the touch operation probability with the motion area to identify the user's potential focus area; The compensation strategy module includes a first strategy unit, which is used to determine a visual compensation strategy for the video stream, and is configured with a light intensity threshold and an acceleration threshold; if the ambient light intensity is lower than the light intensity threshold, the visual compensation strategy for the video stream includes a brightness compensation algorithm; If the lens acceleration is greater than the acceleration threshold, the visual compensation strategy of the video stream includes a blur correction algorithm; If a potential area of interest of the user is identified, the visual compensation strategy of the video stream further includes a local enhancement algorithm for locally enhancing the potential area of interest.

2. The adaptive vision compensation system according to claim 1, wherein: The characteristic data of the video stream include resolution, frame rate, encoding format, degree of geometric distortion, and color offset; The video analysis module includes an information extraction unit, a distortion analysis unit, and a color analysis unit; wherein the information extraction unit is used to extract the resolution, frame rate, and encoding format of the video stream; The distortion analysis unit is used to extract the degree of geometric distortion of the video stream; The color analysis unit is used to extract the color offset of the video stream; The environmental parameters include ambient light intensity, lens acceleration, and operation trajectory data; The environment perception module includes a sensor unit and an operation recording unit; wherein the sensor unit is used to collect ambient light intensity and lens acceleration; The operation recording unit is used to obtain the user's operation trajectory data; the operation trajectory data includes the screen coordinates of each touch operation of the user in the last m seconds; m is a positive integer.

3. The adaptive vision compensation system according to claim 2, wherein: The scene types include industrial detection scenes, medical imaging scenes, and security monitoring scenes; The scene recognition module includes a scene classification unit and a scene verification unit; wherein the scene classification unit calculates the confidence of each scene type based on the video frame; The scene verification unit determines the scene type by performing cross-verification based on the confidence of each scene type and the feature data of the video stream.

4. The adaptive vision compensation system according to claim 3, wherein: The first strategy unit is further configured with a first distortion threshold and a second distortion threshold; if the current scene type is an industrial inspection scene, when the degree of geometric distortion is greater than the first distortion threshold, the visual compensation strategy of the video stream further includes a distortion compensation algorithm; otherwise, when the degree of geometric distortion is greater than the second distortion threshold, the visual compensation strategy of the video stream further includes a distortion compensation algorithm; the first distortion threshold is greater than the second distortion threshold; The first strategy unit is also configured with a first color difference threshold and a second color difference threshold; if the current scene type is a medical imaging scene, when the color offset is greater than the first color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm; otherwise, when the color offset is greater than the second color difference threshold, the visual compensation strategy of the video stream also includes a color correction algorithm; the first color difference threshold is less than the second color difference threshold.

5. The adaptive vision compensation system according to claim 4, wherein: The compensation strategy module further includes a second strategy unit; the second strategy unit optimizes and adjusts the visual compensation strategy based on the change trend of the environmental parameters, specifically including: The second strategy unit is configured with a gradient threshold of light intensity; if the light intensity gradient is greater than the gradient threshold, and the visual compensation strategy includes a brightness compensation algorithm, increasing the iteration frequency of the brightness compensation algorithm; If the lens motion changes suddenly and the visual compensation strategy includes a blur correction algorithm, a multi-frame buffer mechanism is added to the blur suppression algorithm.

6. The adaptive vision compensation system according to claim 5, wherein: The algorithm optimization module includes an algorithm optimization unit; The algorithm optimization unit adaptively selects a vision compensation algorithm based on the vision compensation strategy, specifically including: Obtain an alternative algorithm for each algorithm included in the vision compensation strategy; Generating an algorithm combination of a visual compensation strategy; any group of the algorithm combinations includes an alternative algorithm for each algorithm included in the visual compensation strategy; Set the optimization objective function based on the scenario type; Calculate the optimization target value of each algorithm combination based on the optimization objective function; The algorithm combination with the largest optimization target value is selected and transmitted to the visual compensation unit.

7. The adaptive vision compensation system according to claim 6, wherein: The algorithm optimization module further includes a visual compensation unit; the visual compensation unit performs visual compensation on the video stream based on a visual compensation algorithm, specifically including: Preprocessing the input video stream; the preprocessing includes frame alignment and noise suppression; The visual compensation unit is configured with an alternative algorithm for each algorithm included in the visual compensation strategy; based on the algorithm combination selected by the algorithm optimization unit, the video stream is visually compensated and a video stream with completed visual compensation is output.

8. An adaptive vision compensation method, implemented based on an adaptive vision compensation system according to any one of claims 1 to 7, characterized in that: The following steps are involved: Analyze the real-time input video stream and extract the feature data of the video stream; Collect environmental parameters of the environment corresponding to the video stream; Performing scene recognition based on the environmental parameters and feature data of the video stream to determine the scene type; Predicting the changing trends of the environmental parameters and the user's optimization goals; Determining a visual compensation strategy for the video stream based on the scene type, the changing trend of the environmental parameters, and the user's optimization goal; A visual compensation algorithm is adaptively selected based on the visual compensation strategy, and visual compensation is performed on the video stream based on the visual compensation algorithm.

Images