Visual adaptive method and system based on attention mechanism

Abstract

The invention discloses a visual self-adaption method and system based on an attention mechanism. The method comprises the following steps: acquiring a video picture; based on the attention mechanism, obtaining a key area in the video picture and carrying out amplification processing to obtain an optimized video; and encoding the optimized video, and decoding, playing and displaying the optimized video on a display device. According to the technical scheme provided by the invention, the video picture is adaptively adjusted based on the attention mechanism, and the key area of the picture is amplified or the brightness of the picture is adjusted, so that the redundancy degree of information is effectively reduced, a remote control operator can more conveniently extract key information, accidents are reduced, and the production operation efficiency is improved; free switching can be carried out before automatic adjustment of an algorithm and manual adjustment of a picture by a remote control operator, the operation requirement can be better met, and safety redundancy is added to automatic detection by adding manual switching adjustment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a visual adaptation method and system based on an attention mechanism. Background Art

[0002] The visual system is one of the essential systems for port remote control and automated operations. The port remote control visual system is generally composed of front-end image acquisition equipment, network transmission unit, image display unit, and operation control unit; the image acquisition device is generally a network camera, which encodes the video image and transmits it to the display device through a switch. After the display device completes the decoding operation, it renders the video image on the display; the entire system will complete the switching of the camera image according to the current operation status sent by the control unit. The remote operator controls the port machinery equipment or the automated operation process through the handle or button according to the content of the picture.

[0003] At present, visual systems at home and abroad generally transmit images back to remote control stations through the network, lacking information extraction and processing operations, and relying on remote operators to manually extract key area information from the original images. As the degree of port automation increases, the traditional visual system transmits redundant image information, and at the same time cannot achieve the required resolution for key areas. The main problems are as follows: (1) The zooming of the viewing angle can only focus on the overall picture, and it is impossible to focus on only the key areas; (2) Function triggering can only rely on manual control and cannot be matched with the operation process to achieve automation; (3) The selection of key areas needs to be done manually and cannot be automatically determined based on the image; (4) The zooming operation of key areas is completed on the operating console, which increases the complexity of the display logic. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the current technology, the present invention provides a visual adaptation method based on the attention mechanism, which adaptively adjusts the video picture based on the attention mechanism, amplifies the key areas of the picture or adjusts the brightness, so that the remote control operator can more conveniently extract key information.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions: A visual adaptation method based on an attention mechanism includes the following steps: Get the video screen; Based on the attention mechanism, the key areas in the video are obtained and enlarged to obtain optimized videos; Encode the optimized video and decode and play it on the display device; The method of obtaining the key area in the video screen and amplifying it based on the attention mechanism to obtain the optimized video includes: Based on the deep learning model, feature extraction is performed on the video screen to obtain a feature map; Based on the attention mechanism, the information and features of the key areas are obtained according to the feature map; According to the information and features of the key areas, the images of the key areas are enlarged; The processed key area image is fused with the original video image to obtain an optimized video; Wherein, the information and features of the key areas obtained based on the attention mechanism include: The feature graph is transformed linearly to obtain the query vector, key vector and value vector respectively; Get the attention score based on the query vector and the key vector; Normalize the attention score to obtain the attention weight; The value vector is weighted and summed according to the attention weight to obtain the information and features of the key area.

[0006] According to one aspect of the present invention, the acquiring of video images comprises: Use cameras to collect on-site video images; Transmit the video images to the visual processing center via the network.

[0007] According to one aspect of the present invention, the attention score is obtained by calculating the similarity or correlation between the query vector and the key vector.

[0008] According to one aspect of the present invention, the step of extracting features from a video image based on a deep learning model to obtain a feature map includes: Convolutional neural network is used to extract features from video images and obtain feature maps.

[0009] According to one aspect of the present invention, the visual adaptation method based on the attention mechanism further includes: The screen display strategy is adjusted according to the magnification processing result, and the key area is highlighted in the display screen.

[0010] According to one aspect of the present invention, the step of obtaining a key area in a video screen and performing a magnification process based on an attention mechanism to obtain an optimized video further includes: According to the information of the key area, the brightness of the image in the key area is adjusted.

[0011] According to one aspect of the present invention, the visual adaptation method based on the attention mechanism further includes: When processing the video image, if there is user intervention, the video image will be processed according to the pre-set magnification area rules and the current operating conditions to obtain an optimized video.

[0012] According to one aspect of the present invention, the visual adaptation method based on the attention mechanism further includes: When processing the video images, if a user intervenes, a video processing solution is selected through arbitration.

[0013] According to one aspect of the present invention, selecting a video processing scheme through arbitration includes: Scoring the automatic processing of video images based on the attention mechanism; Scoring the way the video images are processed according to the pre-set magnification area rules and the current working conditions; Select video processing solutions by comparing ratings.

[0014] A visual adaptation system based on an attention mechanism, based on the visual adaptation method based on the attention mechanism as described above, comprising: A collection module, used to obtain video images; The optimization module is used to obtain the key areas in the video screen and enlarge them based on the attention mechanism to obtain the optimized video; The playback module is used to encode the optimized video and decode and play it on the display device.

[0015] Advantages of the present invention: The present invention provides a visual adaptation method based on the attention mechanism, which adaptively adjusts the video picture based on the attention mechanism, amplifies the key areas of the picture or adjusts the brightness, effectively reduces the redundancy of information, enables remote control operators to extract key information more conveniently, reduces the occurrence of accidents, and improves production operation efficiency.

[0016] This method can also switch freely between the automatic adjustment of the algorithm and the manual adjustment of the remote control operator, which can better adapt to the requirements of the operation. Adding manual switching adjustment also adds a safety redundancy to the automatic detection. The manual adjustment method of the operator can be configured according to actual needs, and it is associated with various information such as the current machine's sling height and operation mode, which is more diversified and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is a flow chart of a visual adaptation method based on an attention mechanism according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a visual adaptation method based on an attention mechanism described in Example 2 of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Embodiment 1 like Figure 1 As shown, a visual adaptation method based on an attention mechanism includes the following steps: S1: Get the video image.

[0021] Step S1 includes: Use cameras to collect on-site video images; Transmit the video images to the visual processing center via the network.

[0022] In practical applications, this method is applied to the port remote control vision system, which can use the camera to collect the video images of the port site, transmit them to the vision processing center through the network, process and optimize the video images, and then transmit them to the operation console for display. The vision processing center can be set on the single-machine processing unit of the port quay crane.

[0023] S2: Based on the attention mechanism, the key areas in the video are obtained and enlarged to obtain the optimized video.

[0024] Step S2 includes: S21: Based on the deep learning model, feature extraction is performed on the video image to obtain a feature map.

[0025] In practical applications, deep learning models that can extract features from video images include autoencoders, generative adversarial networks, convolutional neural networks, etc.

[0026] The autoencoder maps the input data into a low-dimensional latent space through the structure of encoder and decoder, and extracts useful features from it.

[0027] The generative adversarial network consists of a generator and a discriminator. In the feature extraction process, the generator can be regarded as a feature extractor, which can generate samples similar to actual data from noise samples and extract useful features. At the same time, the discriminator is used to evaluate the difference between the samples generated by the generator and the actual data, thereby guiding the generator to better extract features.

[0028] In addition, convolutional neural networks can also be used to extract features from video images to obtain feature maps. The specific implementation method is as follows: (1) The convolution layer scans the image through convolution kernels of different sizes to extract local features of the image, such as edges, textures, etc. For example, multiple 3x3 and 5x5 convolution kernels are used to generate multi-layer feature maps.

[0029] (2) The pooling layer downsamples the multi-layer feature map to reduce the data dimension while retaining important feature information. For example, in the maximum pooling operation, the maximum value in the local area is selected as the new feature value. After the multi-layer feature map is reduced in dimension by the pooling layer, the feature map is obtained.

[0030] S22: Based on the attention mechanism, the information and features of the key areas are obtained according to the feature map.

[0031] Step S22 includes: S221: Obtain the query vector Q, key vector K and value vector V by linearly transforming the feature graph respectively.

[0032] Specifically, using different weight matrices W Q , W K and W V Perform linear transformation on the feature graphs to generate query vector Q, key vector K and value vector V: Q = XW Q K = XW K V = XW V Among them, X is the feature map.

[0033] The weight matrix can be learned through the training process of the neural network, or it can be obtained based on prior knowledge or transfer learning of the neural network.

[0034] S222: Obtain an attention score based on the query vector and the key vector.

[0035] In the attention mechanism, the attention score measures the relevance or matching degree between the currently processed target information (query vector Q) and all available information (key vector K).

[0036] The attention score is obtained by calculating the similarity or correlation between the query vector Q and the key vector K, which is usually implemented through a scoring function. Common scoring functions include dot product and additive scoring functions. The dot product scoring function is obtained by calculating the dot product of the query vector Q and the key vector K; while the additive scoring function calculates the similarity through a feedforward neural network.

[0037] S223: Normalize the attention score to obtain the attention weight.

[0038] In practical applications, normalization methods include softmax function, linear normalization, Z-score normalization, batch normalization, etc. Specifically, the method can use the softmax function for normalization to convert the attention score into a probability form, namely, the attention weight, which ensures that the sum of the attention weights of all key vectors K is 1.

[0039] S224: Perform weighted summation on the value vector according to the attention weight to obtain information and features of the key area.

[0040] By using the attention weights to perform weighted summation on the value vector, we can obtain a feature representation containing key information.

[0041] S23: Enlarging the image of the key area according to the information and features of the key area.

[0042] The features after weighted summation in step S224 are magnified by performing a linear transformation on the feature values, such as multiplying by a scaling factor greater than 1, so as to enhance the feature performance of the key area.

[0043] At the same time, according to the position and size information of the key area, the corresponding area in the original image is cropped or enlarged to obtain an enlarged image area.

[0044] Preferably, step S2 further includes: S24: According to the information of the key area, the brightness of the key area image is adjusted.

[0045] The average brightness value of the key area can be calculated, and according to the preset brightness adjustment strategy, a certain brightness increment is added or the brightness is increased proportionally to adjust the pixel brightness of the key area.

[0046] In addition, a brightness conversion function may be used to add a fixed value or multiply a coefficient to the brightness value of each pixel to improve the brightness.

[0047] S25: Fusing the processed key area image with the original video image to obtain an optimized video.

[0048] The processed key area image is fused with the original image, and the enlarged and brightness-adjusted key area is overlaid on the corresponding position of the original image, while other areas remain unchanged.

[0049] In addition, the screen display strategy can be adjusted according to the zoom processing result to highlight the key area in the display screen, such as marking it with a different color frame or giving priority to displaying the key area during display.

[0050] S3: Encode the optimized video and decode and play it on the display device.

[0051] Specifically, step S3 includes: S31: Encode the optimized video to obtain an encoded video.

[0052] The optimized video obtained in step S2 is encoded so that all pictures can be better transmitted in the network. A buffer can be set to cache the optimized video and wait for encoding.

[0053] Video coding refers to the process of converting raw video data into a format suitable for storage or transmission, which usually involves removing redundant information, using the similarity between images, and using compression algorithms to achieve data compression. In practical applications, video can be encoded using coding standards such as the H.26x series, MPEG series, and AVS series.

[0054] S32: Transmit the encoded video to a display device.

[0055] In practical applications, step S2 is usually processed in a visual processing center far away from the display device of the operation console, so the video needs to be transmitted to the operation console through the network for display. Usually, transmission protocols such as RTP, RTSP, HTTP, etc. can be used for transmission.

[0056] S33: Decode the encoded video to obtain a decoded video.

[0057] Video decoding refers to the display device decoding the received encoded data according to the algorithms and standards used during encoding, and converting the decoded data into a playable video format through data reconstruction and image restoration technology.

[0058] S34: Display the decoded video on a display device.

[0059] After the display device completes the decoding operation, it renders the video image on the display for the user to view. If this method is applied to the remote operation and control of port equipment, the operator can view the video image on the console display device and control the port equipment through the handle or buttons according to the image content.

[0060] The beneficial effects of this embodiment are: this method adaptively adjusts the video picture based on the attention mechanism, amplifies the key areas of the picture or adjusts the brightness, effectively reduces the redundancy of information, enables remote control operators to extract key information more conveniently, reduces the occurrence of accidents, and improves production operation efficiency.

[0061] Embodiment 2 like Figure 2 As shown, a visual adaptation method based on an attention mechanism includes the following steps: S1: Get the video image.

[0062] Step S1 includes: Use cameras to collect on-site video images; Transmit the video images to the visual processing center via the network.

[0063] In practical applications, this method is applied to the port remote control vision system, which can use the camera to collect the video images of the port site, transmit them to the vision processing center through the network, process and optimize the video images, and then transmit them to the operation console for display. The vision processing center can be set on the single-machine processing unit of the port quay crane.

[0064] S2: Based on the attention mechanism, the key areas in the video are obtained and enlarged to obtain the optimized video.

[0065] Step S2 includes: S21: Based on the deep learning model, feature extraction is performed on the video image to obtain a feature map.

[0066] In practical applications, deep learning models that can extract features from video images include autoencoders, generative adversarial networks, convolutional neural networks, etc.

[0067] The autoencoder maps the input data into a low-dimensional latent space through the structure of encoder and decoder, and extracts useful features from it.

[0068] The generative adversarial network consists of a generator and a discriminator. In the feature extraction process, the generator can be regarded as a feature extractor, which can generate samples similar to actual data from noise samples and extract useful features. At the same time, the discriminator is used to evaluate the difference between the samples generated by the generator and the actual data, thereby guiding the generator to better extract features.

[0069] In addition, convolutional neural networks can also be used to extract features from video images to obtain feature maps. The specific implementation method is as follows: (1) The convolution layer scans the image through convolution kernels of different sizes to extract local features of the image, such as edges, textures, etc. For example, multiple 3x3 and 5x5 convolution kernels are used to generate multi-layer feature maps.

[0070] (2) The pooling layer downsamples the multi-layer feature map to reduce the data dimension while retaining important feature information. For example, in the maximum pooling operation, the maximum value in the local area is selected as the new feature value. After the multi-layer feature map is reduced in dimension by the pooling layer, the feature map is obtained.

[0071] S22: Based on the attention mechanism, the information and features of the key areas are obtained according to the feature map.

[0072] Step S22 includes: S221: Obtain the query vector Q, key vector K and value vector V by linearly transforming the feature graph respectively.

[0073] Specifically, using different weight matrices W Q , W K and W V Perform linear transformation on the feature graphs to generate query vector Q, key vector K and value vector V: Q = XW Q K = XW K V = XW V Among them, X is the feature map.

[0074] The weight matrix can be learned through the training process of the neural network, or it can be obtained based on prior knowledge or transfer learning of the neural network.

[0075] S222: Obtain an attention score based on the query vector and the key vector.

[0076] In the attention mechanism, the attention score measures the relevance or matching degree between the currently processed target information (query vector Q) and all available information (key vector K).

[0077] The attention score is obtained by calculating the similarity or correlation between the query vector Q and the key vector K, which is usually implemented through a scoring function. Common scoring functions include dot product and additive scoring functions. The dot product scoring function is obtained by calculating the dot product of the query vector Q and the key vector K; while the additive scoring function calculates the similarity through a feedforward neural network.

[0078] S223: Normalize the attention score to obtain the attention weight.

[0079] In practical applications, normalization methods include softmax function, linear normalization, Z-score normalization, batch normalization, etc. Specifically, the method can use the softmax function for normalization to convert the attention score into a probability form, namely, the attention weight, which ensures that the sum of the attention weights of all key vectors K is 1.

[0080] S224: Perform weighted summation on the value vector according to the attention weight to obtain information and features of the key area.

[0081] By using the attention weights to perform weighted summation on the value vector, we can obtain a feature representation containing key information.

[0082] S23: Enlarging the image of the key area according to the information and features of the key area.

[0083] The features after weighted summation in step S224 are magnified by performing a linear transformation on the feature values, such as multiplying by a scaling factor greater than 1, so as to enhance the feature performance of the key area.

[0084] At the same time, according to the position and size information of the key area, the corresponding area in the original image is cropped or enlarged to obtain an enlarged image area.

[0085] Preferably, step S2 further includes: S24: According to the information of the key area, the brightness of the key area image is adjusted.

[0086] The average brightness value of the key area can be calculated, and according to the preset brightness adjustment strategy, a certain brightness increment is added or the brightness is increased proportionally to adjust the pixel brightness of the key area.

[0087] In addition, a brightness conversion function may be used to add a fixed value or multiply a coefficient to the brightness value of each pixel to improve the brightness.

[0088] S25: Fusing the processed key area image with the original video image to obtain an optimized video.

[0089] The processed key area image is fused with the original image, and the enlarged and brightness-adjusted key area is overlaid on the corresponding position of the original image, while other areas remain unchanged.

[0090] In addition, the screen display strategy can be adjusted according to the zoom processing result to highlight the key area in the display screen, such as marking it with a different color frame or giving priority to displaying the key area during display.

[0091] Preferably, the method further comprises: When processing the video image, if there is user intervention, the video image will be processed according to the pre-set magnification area rules and the current operating conditions to obtain an optimized video.

[0092] In practical applications, when this method is applied to the remote operation and control of port equipment, a mechanism for manually zooming in on a certain area of ​​the video screen can also be set, so that the operator can zoom in on a specific area according to their own needs when necessary, and the zoomed-in area can be adaptively adjusted according to the current operating conditions (such as the condition of the spreader, etc.), covering the content automatically detected and processed by the above method. This step is implemented by receiving instructions from the console. The operator triggers the manual zoom option of the current screen through the operating handle or button, thereby realizing the zoom operation of the pre-set area. The specific implementation method is as follows: (1) Command trigger The operator operates the handle or button to trigger the manual zoom option of the current image. At this time, the operation signal is sent to the visual processing center through the operation console.

[0093] (2) Determine the area According to the pre-set magnification area rules, combined with the current working conditions (for example, the current spreader height, working mode, etc.), determine the area that needs to be magnified. These pre-set magnification areas can be set in a variety of ways in the system configuration of the operating console to adapt to the key areas that operators may pay attention to in different working scenarios.

[0094] (3) Screen adjustment According to the determined magnified area, the original video image is cropped to extract the image content of the area.

[0095] The cropped area image is enlarged, for example, by performing algorithms such as interpolation on the image pixels to increase the image size and achieve a local magnification effect. Parameters such as the magnification factor can also be pre-configured or manually adjusted by the operator.

[0096] (4) Screen fusion and display The enlarged area image is fused with the original image, and the enlarged area is covered to the corresponding position of the original image, while keeping other areas unchanged.

[0097] Adjust the screen display strategy according to the processing results, such as using different color boxes to mark the enlarged area or giving priority to the enlarged area when displaying, so that the operator can clearly see the enlarged effect of the key area.

[0098] S3: When processing the video image, if a user intervenes, a video processing solution is selected through arbitration.

[0099] In this method, there are two ways to process the video images: automatic processing of the video images based on the attention mechanism, and processing of the video images based on the intervention of the operator according to the pre-set magnification area rules combined with the current working conditions. In practical applications, these two methods can be arbitrated and one of the video processing solutions can be selected.

[0100] The arbitration method can be to pre-set a scoring function, calculate the score according to the video processing parameters, score the two processing results, and choose the method with a higher score. It is also possible to train a neural network model based on historical data and the pros and cons of the two solutions, use the neural network model to arbitrate the pros and cons, and give a selection plan.

[0101] S4: Encode the optimized video and decode and play it on the display device.

[0102] Specifically, step S4 includes: S41: Encode the optimized video to obtain an encoded video.

[0103] The optimized video obtained in step S2 is encoded so that all pictures can be better transmitted in the network. A buffer can be set to cache the optimized video and wait for encoding.

[0104] Video coding refers to the process of converting raw video data into a format suitable for storage or transmission, which usually involves removing redundant information, using the similarity between images, and using compression algorithms to achieve data compression. In practical applications, video can be encoded using coding standards such as the H.26x series, MPEG series, and AVS series.

[0105] S42: Transmit the encoded video to a display device.

[0106] In practical applications, step S2 is usually processed in a visual processing center far away from the display device of the operation console, so the video needs to be transmitted to the operation console through the network for display. Usually, transmission protocols such as RTP, RTSP, HTTP, etc. can be used for transmission.

[0107] S43: Decode the encoded video to obtain a decoded video.

[0108] Video decoding refers to the display device decoding the received encoded data according to the algorithms and standards used during encoding, and converting the decoded data into a playable video format through data reconstruction and image restoration technology.

[0109] S44: Display the decoded video on a display device.

[0110] After the display device completes the decoding operation, it renders the video image on the display for the user to view. If this method is applied to the remote operation of port equipment, the operator can view the video image on the console display device and control the port equipment through the handle or buttons according to the image content.

[0111] The beneficial effect of this embodiment is that the method can also switch freely between the automatic adjustment of the algorithm and the manual adjustment of the remote control operator, which can better adapt to the requirements of the operation. The addition of manual switching adjustment also adds a safety redundancy to the automatic detection. The manual adjustment method of the operator can be configured according to actual needs, and is associated with various information such as the current machine's sling height and operation mode, which is more diversified and flexible.

[0112] Embodiment 3 A visual adaptation system based on an attention mechanism, based on the visual adaptation method based on the attention mechanism as described in Embodiment 1 or 2, comprising: A collection module, used to obtain video images; The optimization module is used to obtain the key areas in the video screen and enlarge them based on the attention mechanism to obtain the optimized video; The playback module is used to encode the optimized video and decode and play it on the display device.

[0113] Preferably, the system further comprises: The intervention module is used to process the video image. If a user intervenes, the video image will be processed according to the pre-set magnification area rules and the current operating conditions to obtain an optimized video.

[0114] Preferably, the system further comprises: The arbitration module is used to select a video processing solution through arbitration if a user intervenes when processing the video image. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A visual adaptation method based on attention mechanism, characterized in that: The following steps are involved: Get the video screen; Based on the attention mechanism, the key areas in the video are obtained and enlarged to obtain optimized videos; Encode the optimized video and decode and play it on the display device; Wherein, based on the attention mechanism, obtaining the key area in the video screen and enlarging it to obtain the optimized video includes: Based on the deep learning model, feature extraction is performed on the video screen to obtain a feature map; Based on the attention mechanism, the information and features of the key areas are obtained according to the feature map; According to the information and features of the key areas, the images of the key areas are enlarged; The processed key area image is fused with the original video image to obtain an optimized video; Wherein, the information and features of the key areas obtained based on the attention mechanism include: The feature graph is transformed linearly to obtain the query vector, key vector and value vector respectively; Get the attention score based on the query vector and the key vector; Normalize the attention score to obtain the attention weight; The value vector is weighted and summed according to the attention weight to obtain the information and features of the key area.

2. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The obtaining of the video picture comprises: Use cameras to collect on-site video images; Transmit the video images to the visual processing center via the network.

3. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The attention score is obtained by calculating the similarity or correlation between the query vector and the key vector.

4. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The feature extraction of the video image based on the deep learning model to obtain the feature map includes: Convolutional neural network is used to extract features from video images and obtain feature maps.

5. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The visual adaptation method based on the attention mechanism also includes: The screen display strategy is adjusted according to the magnification processing result, and the key area is highlighted in the display screen.

6. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The method of obtaining the key area in the video screen and amplifying it based on the attention mechanism to obtain the optimized video also includes: According to the information of the key area, the brightness of the image in the key area is adjusted.

7. The visual adaptation method based on the attention mechanism according to claim 1, characterized in that: The visual adaptation method based on the attention mechanism also includes: When processing the video image, if there is user intervention, the video image will be processed according to the pre-set magnification area rules and the current operating conditions to obtain an optimized video.

8. The visual adaptation method based on the attention mechanism according to claim 7, characterized in that: The visual adaptation method based on the attention mechanism also includes: When processing the video images, if a user intervenes, a video processing solution is selected through arbitration.

9. The visual adaptation method based on the attention mechanism according to claim 8, characterized in that: The selection of the video processing scheme through arbitration includes: Scoring the automatic processing of video images based on the attention mechanism; Scoring the way the video images are processed according to the pre-set magnification area rules and the current working conditions; Select video processing solutions by comparing ratings.

10. A visual adaptive system based on attention mechanism, characterized in that: The visual adaptation method based on the attention mechanism according to any one of claims 1 to 9 comprises: A collection module, used to obtain video images; The optimization module is used to obtain the key areas in the video screen and enlarge them based on the attention mechanism to obtain the optimized video; The playback module is used to encode the optimized video and decode and play it on the display device.

Images