Video analyzing and processing system based on multi-modal large model and image recognition

Abstract

The invention discloses a video analysis processing system based on a multi-modal large model and image recognition, and relates to the technical field of video analysis, comprising the following steps: acquiring two batches of key frames containing pixel information through a key frame recording module; the sensitive information module identifies a sensitive area by using contour segmentation and deep learning and fuzzifies the sensitive area; the change analysis module compares frame sequence differences, and if the frame sequence differences are different, a visual anchor point compensation technology is used for repairing; and finally, the large model module extracts features and inputs the features into a multi-modal model for analysis and output. Key frame light color features are recovered through a visual anchor point compensation technology, sensitive information is accurately positioned by combining contour segmentation with YOLO, OCR and other technologies, and natural fuzzy processing is performed based on peripheral statistical features; a representative key frame is screened by using a light color fusion coefficient, and a high-dimensional semantic feature is extracted through a pre-training network, so that the analysis efficiency and accuracy of a multi-modal large model are remarkably improved while privacy is protected and visual naturalness is maintained.

Description

Technical Field

[0001] This invention relates to the field of video analysis technology, specifically a video analysis and processing system based on multimodal large models and image recognition. Background Technology

[0002] Traditional methods struggle to fully analyze the multi-dimensional information contained in videos, including images, sounds, and text. Video analysis systems based on multimodal big data models and image recognition technology can simultaneously analyze visual content, voice dialogues, and text information, achieving cross-modal deep semantic understanding and intelligent reasoning. This technology can accurately extract key events and predict potential risks or generate structured summaries, and is widely used in fields such as security monitoring, smart manufacturing, smart healthcare, autonomous driving, and content moderation, effectively improving decision-making efficiency and automation levels.

[0003] Before uploading images to the multimodal large model, they are usually blurred to prevent the leakage of sensitive information in keyframes. However, this operation will significantly change the brightness and color of the keyframe image, causing drastic fluctuations in the brightness and color difference between it and the preceding and following frames. This fluctuation will interfere with the encoder's frame type judgment, causing two types of errors: first, misjudging keyframes that should be used as references as ordinary frames; second, promoting ordinary frames to keyframes. Ultimately, this confusion in frame structure will reduce the encoding quality of the video sequence, resulting in a decrease in the accuracy of the image information extracted and transmitted to the large model. Summary of the Invention

[0004] Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a video analysis and processing system based on multimodal large models and image recognition. This system solves the problem that blurring keyframes interferes with the encoder's frame type judgment, causing video structure chaos and ultimately reducing the quality of image information uploaded to large models.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a video analysis and processing system based on multimodal large model and image recognition, comprising the following specific modules: a keyframe image recording module: performing a first keyframe image acquisition on the video data to obtain a first keyframe image, which includes the number of pixels, pixel coordinates, pixel RGB color values, and pixel brightness values; after receiving the return instruction from the sensitive information blurring module, performing a second keyframe image acquisition on the video data to obtain a second keyframe image, and then executing the keyframe image change analysis module; and a sensitive information blurring module: performing image contour segmentation and tracking algorithms on the first keyframe image... The keyframe image is processed to obtain the image contour. A trained deep neural network model is used to scan and analyze the image contour to obtain the sensitive contour. The sensitive contour is then blurred to obtain the blurred contour, and the result is returned to the keyframe image recording module. The keyframe image change analysis module compares the frame order of the first keyframe image and the second keyframe image. If the frame order is the same, the large model output module is executed. If the frame order is different, the blurred first keyframe image is repaired using visual anchor point steady-state compensation technology, and the comparison is repeated. The large model output module extracts information from the first keyframe image, analyzes and processes it using a multimodal large model, outputs the results, and then terminates the process.

[0006] Further, the specific method for obtaining the first keyframe image is as follows: the brightness value and RGB color value in each frame image are comprehensively calculated and standardized to obtain the light and color fusion coefficient. The difference between the light and color fusion coefficient of the i-th frame image and the light and color fusion coefficient of the (i+1)-th frame image is calculated and the absolute value is taken to obtain the light and color fusion difference value. This process is repeated until the light and color fusion coefficients of any two adjacent frames are calculated. A preset light and color fusion difference threshold is set, and the light and color fusion difference values ​​within the light and color fusion difference threshold are selected. The next frame image corresponding to the light and color fusion difference value is defined as the first keyframe image.

[0007] Furthermore, the specific method for obtaining the light and color fusion coefficient is as follows: convert the RGB color value of the pixel into the gray value of the pixel, calculate the average gray value of the pixel based on the number of pixels and the gray value of the pixel in each frame of the image, calculate the average brightness value based on the number of pixels and the brightness value of the pixel in each frame of the image, and multiply the average brightness value and the average gray value of the pixel to obtain the light and color fusion coefficient.

[0008] Furthermore, the specific method for obtaining the blurred contour is as follows: the center point of the sensitive contour is calculated based on the number of pixels and the coordinates of the pixels within the sensitive contour; the pixels adjacent to the edge pixels of the sensitive contour are comprehensively analyzed with the center point of the sensitive contour to obtain sample pixels; and the sensitive contour area is filled with sample pixels to obtain the blurred contour.

[0009] Furthermore, the specific method for obtaining the specimen pixels is as follows: the edge pixels and the center point of the sensitive contour are sequentially calculated using Euclidean distance to obtain a target distance group; the pixels adjacent to the edge pixels of the sensitive contour and the center point of the sensitive contour are sequentially calculated using Euclidean distance to obtain a detection distance group; the detection distances in the detection distance group are sequentially compared with the target distances in the target distance group, and the pixels whose target distances are greater than the detection distances are selected and recorded as specimen pixels.

[0010] Furthermore, the specific method for obtaining the blurred contour is as follows: the proportion of grayscale value and brightness value of each sample pixel in the sample pixels is calculated respectively, and the grayscale value and brightness value of the pixel with the largest proportion are recorded as the coverage grayscale value and coverage brightness value respectively. Then, the grayscale value of each pixel in the sensitive contour is set as the coverage pixel grayscale value, and the brightness value of each pixel in the sensitive contour is set as the coverage brightness value.

[0011] Furthermore, the specific method for repairing the blurred first keyframe image using visual anchor point steady-state compensation technology in the keyframe image change analysis module is as follows: calculate the four corner pixels in the first keyframe image, and adjust the grayscale value and brightness value respectively starting from the four corner pixels. Preset grayscale adjustment threshold and brightness adjustment threshold. If the adjusted grayscale value is within the grayscale adjustment threshold, stop adjusting. Similarly, if the adjusted brightness value is within the brightness adjustment threshold, stop adjusting. Otherwise, randomly distribute it to adjacent pixels, and so on, until the light and color fusion coefficient of the blurred first keyframe image is equal to the light and color fusion coefficient of the unblurred first keyframe image, and convert the grayscale image back to the original color image.

[0012] Furthermore, the specific calculation method for the four corner pixels is as follows: the center point of the first keyframe is calculated based on the number of pixels and the coordinates of the pixels in the first keyframe image. The remaining pixels are then sequentially calculated with respect to the center point of the first keyframe using Euclidean distance to obtain a value distance group, which includes various distances. The four largest distances are selected from the value distance group, and the corresponding pixels are the corner pixels.

[0013] Furthermore, the specific method for adjusting the grayscale and brightness values ​​is as follows: compare the light and color fusion coefficient of the first keyframe image before blurring with the light and color fusion coefficient of the first keyframe image after blurring. If the light and color fusion coefficient of the first keyframe image before blurring is greater than the light and color fusion coefficient of the first keyframe image after blurring, then increase the grayscale or brightness value; if the light and color fusion coefficient of the first keyframe image before blurring is less than the light and color fusion coefficient of the first keyframe image after blurring, then decrease the grayscale or brightness value.

[0014] Beneficial effects Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects: 1. By using visual anchor point steady-state compensation technology, the grayscale and brightness values ​​are iteratively adjusted from the four corners of the image to accurately restore the light and color fusion coefficients after blurring to their original values. This ensures that key frame determination is not interfered with by sensitive information processing. As a result, images that truly carry significant changes in scene or action can be stably extracted and input into the multimodal large model, effectively avoiding the problem of decreased analysis accuracy caused by frame structure disorder.

[0015] 2. By employing image contour segmentation and tracking algorithms to accurately capture image contours, and leveraging YOLO object detection, OCR, and named entity recognition technologies, we can precisely locate sensitive information such as faces, license plates, documents, specific items, and text in multiple dimensions. Subsequently, we fill the area with the statistical mode of the pixels surrounding the sensitive contour, which is the grayscale and brightness value with the largest proportion. This not only completely blurs sensitive details but also ensures that the filled area is consistent with the surrounding environment in terms of visual statistical features, reducing the risk of visual conflict and balancing privacy protection with a natural visual experience.

[0016] 3. By filtering out keyframes with high differences through light and color fusion coefficients, core semantic information is preserved with minimal encoding, avoiding repeated analysis of similar content. This keyframe extraction mechanism based on brightness and color fusion can accurately capture significant changes in the scene or action, ensuring that the features of the input multimodal large model are most representative. Compared with frame-by-frame processing, it significantly reduces the amount of model input data, effectively reducing computing power consumption and response latency while ensuring analysis quality.

[0017] 4. The high-dimensional semantic feature vector extracted by the pre-trained deep vision network has a distribution that is highly consistent with the original keyframes. This feature serves as the standardized input for the multimodal large model, supporting cross-modal alignment and understanding. The visual anchor compensation mechanism controls the contrast difference between pixels and surrounding areas by adjusting the threshold through preset grayscale or brightness values, avoiding visual abrupt changes, and restoring the grayscale image to a color image. The complete preservation of the color dimension further enhances visual realism and information recognition, ensuring that the input received by the large model is consistent and reliable both semantically and visually.

[0018] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0019] Figure 1 This invention is a flowchart of a video analysis and processing method based on multimodal large models and image recognition.

[0020] Figure 2 This invention is a structural diagram of a video analysis and processing system based on multimodal large models and image recognition. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0023] Example 1: like Figures 1-2 As shown, this embodiment of the invention provides a video analysis and processing system based on multimodal large models and image recognition, comprising the following specific modules: Keyframe image recording module: The video data is captured for the first keyframe image to obtain the first keyframe image, which includes the number of pixels, pixel coordinates, pixel RGB color value and pixel brightness value. After receiving the return instruction from the sensitive information blurring module, the video data is subjected to a second keyframe image acquisition to obtain the second keyframe image. The specific acquisition method of the second keyframe image is the same as that of the first keyframe image. Then, the keyframe image change analysis module is executed.

[0024] Sensitive information blurring module: The first keyframe image is processed by image contour segmentation and tracking algorithm to obtain image contour. Image contour segmentation algorithm: such as Sobel algorithm, firstly, 3×3 convolution kernels in the horizontal and vertical directions are convolved to obtain horizontal and vertical gradient matrices; then the gradient magnitude and direction are calculated, the former determines the edge and the latter determines the edge extension direction; finally, through threshold binarization, pixels above the threshold are marked as contours. Image contour tracking algorithms: For example, the Lucas-Kanade optical flow method establishes optical flow constraint equations based on the assumption of constant brightness between adjacent frames, calculates spatial and temporal gradients through the Sobel algorithm and frame difference, constructs an overdetermined set of equations assuming consistent neighborhood motion, solves the optical flow velocity using the least squares method, and finally performs tracking and position updates, thereby enabling the tracking of image contours. The trained deep neural network model is used to scan and analyze image contours to obtain sensitive contours. For example, the YOLO object detection model is used to locate sensitive image contours, such as face contours, license plate contours, document contours, and specific object contours. At the same time, for text information, optical character recognition technology is combined to convert the text in the image contours into readable text, and named entity recognition in natural language processing is used to determine whether the image contours are contours of sensitive data such as names, addresses, and ID card numbers. The sensitive contours are blurred to obtain blurred contours, which are then returned to the keyframe image recording module.

[0025] Keyframe image change analysis module: The frame order of the first keyframe image and the second keyframe image is compared. If the frame order is the same, it means that the first keyframe image and the second keyframe image are the same keyframe images, and then the large model output module is executed; if the frame order is different, it means that the first keyframe image and the second keyframe image are different keyframe images, and then the blurred first keyframe image is repaired by visual anchor point steady-state compensation technology and compared again.

[0026] Large model output module: Information is extracted from the first keyframe image through a pre-trained deep vision network. The hierarchical representation of edges, textures, parts and complete objects is abstracted step by step through multiple convolutions and activation functions. High-dimensional semantic feature vectors are output in fully connected or pooling layers as the standard input of the multimodal large model to support cross-modal alignment and understanding. Finally, the multimodal large model analyzes and processes the data and outputs the results.

[0027] Example 2 differs from Example 1 in that: The specific method for obtaining the first keyframe image is as follows: The video data is composed of different frame images. The brightness value and RGB color value in each frame image are comprehensively calculated and standardized to eliminate the difference in dimensions and transform the values ​​of different orders of magnitude into a unified numerical range to obtain the light and color fusion coefficient. The light and color fusion coefficient of the i-th frame image and the light and color fusion coefficient of the (i+1)-th frame image are calculated and the absolute value is taken to obtain the light and color fusion difference value. This process is repeated until the light and color fusion coefficients of any two adjacent frames have been calculated. A preset light and color fusion difference threshold is set and is in the form of an interval. Light and color fusion difference values ​​within the light and color fusion difference threshold are selected. Light and color fusion difference values ​​within this threshold are all relatively large. The next frame image corresponding to the light and color fusion difference value is defined as the first key frame image. When analyzing video data using a multimodal large model, the high redundancy and computational cost of analyzing complete video data necessitate using keyframe images. This approach preserves core information with minimal encoding, significantly reducing processing costs and computational overhead. Selecting frames with large differences in light and color fusion as keyframes accurately captures significant changes in the scene or actions, avoiding redundant analysis of similar content and ensuring that the extracted features are the most representative and discriminative, thereby improving overall efficiency and accuracy.

[0028] The specific method for obtaining the light and color fusion coefficient is as follows: Since the RGB color value of a pixel includes the color values ​​of the red channel, green channel, and blue channel, in order to reduce the computational overhead and improve the computational efficiency, each frame of the image is converted into a grayscale image. That is, the original three-channel color values ​​are converted into a single grayscale channel color value, resulting in the pixel grayscale value, which ranges from 0 to 255. The average pixel gray value is calculated based on the number of pixels and the pixel gray value in each frame of the image. The average brightness value is calculated based on the number of pixels and the pixel brightness value in each frame of the image. The average brightness value and the average pixel gray value are multiplied to obtain the light and color fusion coefficient. The multiplication operation amplifies the common change trend of the average pixel gray value and the average brightness value, making the light and color fusion coefficient more sensitive to significant differences in the scene or action. ; in, Indicates the light and color blending coefficient. Indicates the number of pixels. Indicates the average brightness value. This represents the average grayscale value of each pixel.

[0029] The specific method for obtaining the blurred outline is as follows: The center point of the sensitive contour is calculated based on the number and coordinates of the pixels within the sensitive contour. The pixels adjacent to the edge pixels of the sensitive contour are then analyzed together with the center point of the sensitive contour to obtain the sample pixels, which are the adjacent pixels outside the sensitive contour. The sensitive contour area is then filled with the sample pixels to obtain the blurred contour.

[0030] The specific method for obtaining the specimen pixels is as follows: The Euclidean distance between edge pixels and the center point of the sensitive contour is calculated sequentially to obtain the target distance group. The Euclidean distance represents the straight-line distance between two pixels. The Euclidean distance between the pixels adjacent to the edge pixels of the sensitive contour and the center point of the sensitive contour is calculated sequentially to obtain the detection distance group. The detection distance in the detection distance group is compared with the target distance in the target distance group. Pixels whose target distance is greater than the detection distance are selected as adjacent pixels outside the sensitive contour and are recorded as specimen pixels. This is because the pixels adjacent to the sensitive contour can be either outside the edge or inside the edge.

[0031] The specific method for obtaining the blurred contour is as follows: The proportion of grayscale value and brightness value of each pixel in the specimen is calculated separately. Then, a fast sorting algorithm is used to record the grayscale value and brightness value of the pixel with the largest proportion as the covering grayscale value and covering brightness value, respectively. This increases the probability of visual consistency between the processed sensitive contour area and the surrounding area and reduces the risk of visual conflict. Note that the grayscale value and brightness value of the pixel with the largest proportion may not correspond to the same pixel in the specimen. Then, the grayscale value of each pixel in the sensitive contour is set as the covering pixel grayscale value, and the brightness value of each pixel in the sensitive contour is set as the covering brightness value.

[0032] The specific method for repairing the blurred first keyframe image using visual anchor point steady-state compensation technology is as follows: Because the grayscale and brightness values ​​of a portion of the first keyframe image are blurred, the light and color fusion coefficient changes, causing the light and color fusion difference between this keyframe and the previous or next frame to decrease or increase. This results in a keyframe that originally met the light and color fusion difference threshold being judged as an ordinary frame due to the decrease in the light and color fusion difference, or a subsequent ordinary frame being misjudged as a keyframe due to the increase in the light and color fusion difference. This leads to distortion in keyframe extraction, making it impossible to effectively transmit images of real scenes or significantly changed actions to the multimodal large model, thereby reducing its analysis accuracy. Therefore, four corner pixels are calculated in the first keyframe image. The corner pixels are selected as the starting point for adjustment because they are far from the visual center and have the lowest information density, thus having the least impact on the core semantics of the image. The adjustment amount is gradually propagated from the edge to the center, which can accurately restore the light and color fusion coefficient at the lowest cost, and avoid directly disturbing the key visual area, ensuring that the compensation process is smooth and controllable. Starting from the four corner pixels, the grayscale and brightness values ​​are adjusted separately. Preset grayscale and brightness adjustment thresholds, in the form of intervals, are used to control the contrast difference between the adjusted pixel and its surrounding pixels. If the adjusted grayscale value is within the grayscale adjustment threshold, the adjustment stops; similarly, if the adjusted brightness value is within the brightness adjustment threshold, the adjustment stops. Otherwise, the value is randomly distributed to adjacent pixels, and so on, until the color fusion coefficient of the first keyframe image after blurring equals that of the first keyframe image before blurring. This not only ensures the natural smoothness of local adjustments and avoids visual abrupt changes through threshold control, but also globally guarantees that the repaired frame is consistent with the original frame in terms of color statistical characteristics. This maintains the encoder's stable recognition of keyframes and ensures that the input received by the subsequent multimodal large model is visually and semantically reliable. Furthermore, converting the grayscale image back to the original color image restores the color dimension, enhances visual realism and information recognition, and facilitates subsequent multimodal large model analysis.

[0033] The specific calculation method for the four corner pixels is as follows: The center point of the first keyframe is calculated based on the number and coordinates of the pixels in the first keyframe image. The remaining pixels are then sequentially subjected to Euclidean distance calculations with respect to the center point of the first keyframe to obtain a value distance group, which includes various distances. The four largest distances are selected from the value distance group using a quick sorting algorithm. The pixels corresponding to these distances are the corner pixels, meaning that the image is a symmetrical rectangle and the distances from the four corner pixels to the center point of the first keyframe are equal.

[0034] The specific methods for adjusting grayscale and brightness values ​​are as follows: The light and color fusion coefficients of the first keyframe image before blurring are compared with those of the first keyframe image after blurring. If the light and color fusion coefficients of the first keyframe image before blurring are greater than those of the first keyframe image after blurring, the grayscale value or brightness value is increased; if the light and color fusion coefficients of the first keyframe image before blurring are less than those of the first keyframe image after blurring, the grayscale value or brightness value is decreased. Because the light and color fusion coefficient quantifies the overall brightness and grayscale of an image, the blurring disturbance causes it to deviate from the original value. When the light and color fusion coefficient of the first keyframe image before blurring is greater than that of the first keyframe image after blurring, it indicates that the blurring operation weakens the light and color intensity, and the gain needs to be increased in reverse to compensate for the attenuation; otherwise, it should be decreased to suppress overshoot.

[0035] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A video analysis processing system based on multi-modal large model and image recognition, characterized in that: The specific modules include the following: The key frame image recording module: the first key frame image is collected from the video data, including the number of pixel points, pixel point coordinates, pixel point RGB color values and pixel point brightness values; After receiving the return instruction of the sensitive information blurring processing module, the second key frame image is collected from the video data, and the key frame image change analysis module is executed again; The sensitive information blurring processing module: the first key frame image is processed by the image contour segmentation and tracking algorithm to obtain the image contour, the trained deep neural network model is used to scan and analyze the image contour to obtain the sensitive contour, the sensitive contour is blurred to obtain the blurred contour, and the key frame image recording module is returned; The key frame image change analysis module: the frame sequence of the first key frame image and the second key frame image is compared, if the frame sequence is the same, the large model output module is executed, if the frame sequence is different, the blurred first key frame image is repaired by the visual anchor point steady compensation technology, and the comparison is re-performed; The large model output module: the first key frame image is information extracted, and the multi-modal large model is analyzed and processed to output the result and end. 2.The video analysis processing system based on multi-modal large model and image recognition according to claim 1, wherein: The specific acquisition method of the first key frame image is as follows: The brightness value and the RGB color value in each frame of image are comprehensively calculated and standardized to obtain the light color fusion coefficient, the light color fusion coefficient of the i-th frame of image and the light color fusion coefficient of the i+1-th frame of image are difference calculated and the absolute value is taken to obtain the light color fusion difference value, and the process is repeated until the light color fusion coefficients of any two adjacent frames of image are calculated, a preset light color fusion difference threshold is set, the light color fusion difference values within the light color fusion difference threshold are screened out, and the next frame of image corresponding to the light color fusion difference value is defined as the first key frame image. 3.The video analysis processing system based on multi-modal large model and image recognition according to claim 2, characterized in that: The specific acquisition method of the light color fusion coefficient is as follows: The pixel point RGB color value is converted into pixel point gray value, the average pixel point gray value is calculated according to the pixel point number and the pixel point gray value in each frame of image, the average brightness value is calculated according to the pixel point number and the pixel point brightness value in each frame of image, the product of the average brightness value and the average pixel point gray value is calculated to obtain the light color fusion coefficient. 4.The video analysis processing system based on multi-modal large model and image recognition according to claim 3, characterized in that: The specific acquisition method of the blurred contour is as follows: The center point of the sensitive contour is calculated according to the pixel point number and the pixel point coordinates in the sensitive contour, the specimen pixel points are obtained by comprehensively analyzing the pixel points adjacent to the edge pixel points of the sensitive contour and the center point of the sensitive contour, and the sensitive contour area is filled with the specimen pixel points to obtain the blurred contour. 5.The video analysis processing system based on multi-modal large model and image recognition according to claim 4, characterized in that: The specific acquisition method of the specimen pixel point is as follows: The target distance group is obtained by sequentially calculating the Euclidean distance between the edge pixel points and the center point of the sensitive contour, the detection distance group is obtained by sequentially calculating the Euclidean distance between the pixel points adjacent to the edge pixel points of the sensitive contour and the center point of the sensitive contour, the detection distance in the detection distance group is compared with the target distance in the target distance group, and the pixel points with the target distance greater than the detection distance are screened out and recorded as specimen pixel points. 6.The video analysis processing system based on multi-modal large model and image recognition according to claim 5, characterized in that: The specific acquisition method of the blurred contour is as follows: The proportion of the pixel point gray value and the pixel point brightness value in each sample pixel point is counted respectively, and the pixel point gray value and the pixel point brightness value with the largest proportion are recorded as the covering gray value and the covering brightness value respectively, and the gray value of each pixel point in the sensitive contour is set as the covering pixel point gray value, and the brightness value of each pixel point in the sensitive contour is set as the covering brightness value.

7. The video analysis processing system based on multi-modal large model and image recognition according to claim 6, characterized in that: The specific way in which the first key frame image blurred by the visual anchor point steady-state compensation technology is repaired in the key frame image change analysis module is: Four corner pixel points are calculated in the first key frame image, and the gray value and the brightness value are adjusted from the four corner pixel points, preset gray adjustment threshold and brightness adjustment threshold, if the adjusted gray value is within the gray adjustment threshold, the adjustment is stopped, and similarly, if the adjusted brightness value is within the brightness adjustment threshold, the adjustment is stopped, otherwise, the adjacent pixel points are randomly divided, and so on, until the light color fusion coefficient of the first key frame image after the blur processing is equal to the light color fusion coefficient of the first key frame image before the blur processing, and the gray image is converted back to the original color image. 8.The video analysis processing system based on multi-modal large model and image recognition according to claim 7, characterized in that: The specific calculation method of the four corner pixel points is: According to the number and coordinates of the pixel points in the first key frame image, the first key frame center point is calculated, and the remaining pixel points are sequentially calculated with the first key frame center point by the Euclidean distance to obtain the value distance group, which includes various distances, and the four largest distances are selected in the value distance group, and the corresponding pixel points of the four largest distances are the corner pixel points. 9.The video analysis processing system based on multi-modal large model and image recognition according to claim 8, characterized in that: The specific way of adjusting the gray value and the brightness value is: The light color fusion coefficient of the first key frame image before the blur processing is compared with the light color fusion coefficient of the first key frame image after the blur processing, if the light color fusion coefficient of the first key frame image before the blur processing is greater than the light color fusion coefficient of the first key frame image after the blur processing, the gray value or the brightness value is adjusted to be larger; If the light color fusion coefficient of the first key frame image before the blur processing is less than the light color fusion coefficient of the first key frame image after the blur processing, the gray value or the brightness value is adjusted to be smaller.

Images