Image data analysis and processing method and system based on liquid lens

Abstract

The present invention provides an image data analysis and processing method and system based on a liquid lens, belonging to the field of image processing technology, comprising: S1: based on massive historical monitoring videos acquired by the liquid lens, analyzing the concentrated distribution area of the monitoring object in the monitoring video frame area of the liquid lens monitoring scene; S2: calculating the defocus data of the monitoring object in each historical monitoring video frame; S3: partitioning and focusing on repairing the historical monitoring video frame based on the defocus data to obtain the repaired historical monitoring video frame; based on the concentrated distribution area of the monitoring object and the defocus degree of the monitoring area corresponding to the monitoring object, the video frame acquired based on the liquid lens is partitioned and focused on repairing, which not only ensures that the repaired image can meet the display requirements, but also improves the repair efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for analyzing and processing image data based on a liquid lens. Background Art

[0002] Currently, there are a variety of traditional mechanical cameras on the market, including fixed-focus cameras, mechanical zoom cameras, and panoramic spherical surveillance fisheye wide-angle cameras. Each traditional mechanical camera has certain limitations, so common monitoring systems often use multiple different cameras to compensate for functional deficiencies. However, while this multi-camera solution can capture complex road information, the overall system is overly complex and inevitably has some blind spots. The processing volume of this solution is also very large, which is not conducive to compatibility and use with artificial intelligence (AI) such as facial recognition and big data algorithms. Liquid lenses, which modulate current to control the deformation of the liquid inside the device, thereby changing the lens' focal length and enabling clear imaging of objects at varying distances, offer advantages over traditional mechanical lenses, such as fast focusing, small size, and simple structure.

[0003] However, for images directly acquired by liquid lenses, some monitored objects may be out of focus. For these out-of-focus images, there is a lack of efficient image data analysis and processing methods to efficiently repair the out-of-focus monitored objects in the images acquired by liquid lenses so that they meet display requirements.

[0004] Therefore, the present invention proposes an image data analysis and processing method and system based on a liquid lens. Summary of the Invention

[0005] The present invention provides a liquid lens-based image data analysis and processing method and system, which is used to partition and focus repair of video frames acquired based on the liquid lens based on the concentrated distribution area of the monitored object and the defocus degree of the monitored area corresponding to the monitored object, thereby ensuring that the repaired image can meet the display requirements and improving the repair efficiency.

[0006] The present invention provides an image data analysis and processing method based on a liquid lens, comprising:

[0007] S1: Based on the massive amount of historical surveillance videos acquired by the liquid lens, the concentrated distribution area of the surveillance objects in the surveillance scene of the liquid lens is analyzed in the surveillance video frame area;

[0008] S2: Calculate the defocus data of the monitored object in each historical monitoring video frame;

[0009] S3: Performing partition-based key restoration on the historical surveillance video frames based on the defocus data to obtain restored historical surveillance video frames.

[0010] Preferably, the image data analysis and processing method based on the liquid lens further includes:

[0011] S4: Analyze the rough focus range of the liquid lens based on the concentrated distribution area;

[0012] S5: Tracking and predicting the trajectories of all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens, and determining the predicted movement trajectories of all the latest monitored objects in the monitoring video frames;

[0013] S6: Predicting a focus position movement trajectory within a rough focus range based on the predicted movement trajectory;

[0014] S7: Pre-focusing the liquid lens based on the focus position movement trajectory to obtain a focus control result of the liquid lens.

[0015] Preferably, the image data analysis and processing method based on the liquid lens, S1: based on the massive historical monitoring video acquired by the liquid lens, analyzing the concentrated distribution area of the monitoring objects in the monitoring scene of the liquid lens in the monitoring video frame area, including:

[0016] Determine the area where the monitored object is located in the monitoring scene of the liquid lens in each historical monitoring video frame of the massive historical monitoring video acquired by the liquid lens, and use the area as the monitoring area;

[0017] The ratio of the number of times each pixel position in the monitoring video frame area exists in all monitoring areas to the total number of historical monitoring video frames is regarded as the frequency of the monitoring object at the corresponding pixel position;

[0018] The pixel positions in the monitoring video frame area with a frequency not less than the concentration frequency threshold are regarded as concentrated distribution positions, and all concentrated distribution positions are summarized to obtain the concentrated distribution area of the monitoring object in the monitoring scene of the liquid lens in the monitoring video frame area.

[0019] Preferably, the image data analysis and processing method based on the liquid lens, S2: calculating the defocus data of the monitored object in each historical monitoring video frame, includes:

[0020] Determine the focus pixel position in the historical surveillance video frame based on pixel gradient information in the historical surveillance video frame;

[0021] The distance between each pixel position and the focus pixel position in the monitoring area corresponding to the monitoring object in each historical monitoring video frame is regarded as the defocus data of the monitoring object in the historical monitoring video frame.

[0022] Preferably, the image data analysis and processing method based on the liquid lens determines the focus pixel position in the historical monitoring video frame based on the pixel gradient information in the historical monitoring video frame, including:

[0023] Determine all pixel point sequences in each preset direction of the historical surveillance video frame, and generate a pixel gradient sequence for all pixel point sequences in each preset direction based on the difference in pixel values of adjacent pixel points in the pixel point sequence in the historical surveillance video frame;

[0024] Determine a partial focus range dividing boundary between adjacent pixel points corresponding to the maximum pixel value difference in all pixel gradient sequences;

[0025] The boundaries of all partial focus ranges are connected to form the focus range of the historical monitoring video frame, and the center position of the focus range is used as the focus pixel position of the historical monitoring video frame.

[0026] Preferably, the image data analysis and processing method based on the liquid lens, S3: performing partitioned and focused restoration of the historical surveillance video frames based on the defocus data to obtain the restored historical surveillance video frames, comprises:

[0027] The ratio between the distance between each pixel position and the focused pixel position in the monitoring area in the historical monitoring video frame in the out-of-focus data and the maximum value of the distances between all pixel positions and the focused pixel position in the monitoring area is used as the first repair weight of the corresponding pixel in the monitoring area;

[0028] Determine the center position of the monitoring area, and use the ratio of the distance between each pixel position in the monitoring area and the center position of the monitoring area to the maximum value of the distances between all pixel positions in the monitoring area and the center position of the monitoring area as the second restoration weight of the corresponding pixel in the monitoring area;

[0029] The difference between the first restoration weight and the second restoration weight of the pixel in the monitoring area is regarded as the final restoration weight of the corresponding pixel;

[0030] The historical monitoring video frames with the final restoration weights of all pixels in the monitoring area marked are input into the video frame partition key restoration model to obtain the restored historical monitoring video frames.

[0031] Preferably, the image data analysis and processing method based on the liquid lens, S4: analyzing the rough focus range of the liquid lens based on the concentrated distribution area, includes:

[0032] Effective focusing radius based on preset requirements;

[0033] Determine the tangent line of the concentrated distribution area at each contour point of the concentrated distribution area, and use the straight line passing through the corresponding contour point and perpendicular to the corresponding tangent line as the assumed inner diameter straight line of the concentrated distribution area at the corresponding contour point;

[0034] Assuming that the distance between the inner diameter straight line and the corresponding contour point is the effective focusing radius, the position located inside the concentrated distribution area is regarded as the outermost edge focusing center position of the corresponding contour point;

[0035] The area enclosed by connecting the outermost focus center positions of all contour points in the concentrated distribution area is regarded as the rough focus range of the liquid lens.

[0036] Preferably, the image data analysis and processing method based on the liquid lens, S5: tracking and predicting the trajectories of all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens, and determining the predicted movement trajectories of all the latest monitored objects in the monitoring video frame, includes:

[0037] Track all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens to obtain the current tracking trajectories of all the latest monitored objects;

[0038] Based on the current tracking trajectory, the category of the latest monitored object and the preset trajectory prediction model, the predicted movement trajectory of all the latest monitored objects in the monitored video frame is determined.

[0039] Preferably, the image data analysis and processing method based on the liquid lens, S6: predicting the focus position movement trajectory within the rough focus range based on the predicted movement trajectory, includes:

[0040] The predicted movement trajectories of all the latest monitored objects in the latest historical monitoring video are averaged in the same video frame to obtain the focus position of the corresponding video frame;

[0041] The focus positions of all video frames are sorted and fitted according to the video frame order to obtain the initial focus position movement trajectory;

[0042] The portion of the initial focus position movement trajectory outside the rough focus range is deleted to obtain the focus position movement trajectory.

[0043] The present invention proposes an image data analysis and processing system based on a liquid lens, comprising:

[0044] The concentrated area analysis module is used to analyze the concentrated distribution area of the monitored objects in the monitoring scene of the liquid lens in the monitoring video frame area based on the massive historical monitoring videos acquired by the liquid lens;

[0045] A defocus data calculation module is used to calculate the defocus data of the monitored object in each historical monitoring video frame;

[0046] The video frame repair module is used to perform partition-focused repair on historical surveillance video frames based on defocus data to obtain repaired historical surveillance video frames.

[0047] The beneficial effects of the present invention compared to the prior art are: based on the concentrated distribution area of the monitored object and the defocus degree of the monitored area corresponding to the monitored object, the video frames obtained based on the liquid lens are partitioned and focused on repair, which not only ensures that the repaired image can meet the display requirements, but also improves the repair efficiency.

[0048] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 This is a flow chart of an image data analysis and processing method based on a liquid lens in an embodiment of the present invention;

[0052] Figure 2 Flowchart of another method for analyzing and processing image data based on a liquid lens in an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of an image data analysis and processing system based on a liquid lens in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0055] Example 1:

[0056] The present invention provides an image data analysis and processing method based on a liquid lens, referring to Figure 1 ,include:

[0057] S1: Based on a large amount of historical surveillance videos acquired by the liquid lens (i.e., surveillance videos previously acquired by the liquid lens), analyzing concentrated distribution areas (i.e., areas where the surveillance objects are concentrated in the surveillance video frame area) of the liquid lens surveillance scene (i.e., the actual spatial scene monitored by the liquid lens) (i.e., objects monitored by the liquid lens, such as pedestrians and vehicles in traffic monitoring);

[0058] S2: Calculate the defocus data of the monitored object in each historical monitoring video frame (i.e., the video frame in the historical monitoring video) (i.e., the deviation data of the position of the monitored object in the historical monitoring video frame relative to the focus position in the historical monitoring video frame);

[0059] S3: Based on the defocus data, the historical surveillance video frame is partitioned and focused on repairing (i.e., the area where the monitored object is located in the historical surveillance video frame is focused on repairing), and a repaired historical surveillance video frame is obtained (i.e., the repaired historical surveillance video frame).

[0060] Based on the concentrated distribution area of the monitored object and the defocus degree of the monitored area corresponding to the monitored object, the video frames obtained based on the liquid lens are partitioned and repaired in a focused manner, which not only ensures that the repaired image can meet the display requirements (for example, the clarity of the area where the monitored object is located needs to reach a preset clarity threshold or the sharpness of the area where the monitored object is located needs to reach a preset sharpness threshold), but also improves the repair efficiency.

[0061] Example 2:

[0062] On the basis of Example 1, the image data analysis and processing method of the liquid lens is as follows: Figure 2 , also includes:

[0063] S4: Analyzing a rough focus range of the liquid lens based on the concentrated distribution area (i.e., a rough range of a suitable focus position preliminarily determined in the surveillance video frame area in order to ensure that the surveillance object has a satisfactory display effect in the surveillance video frame obtained based on the liquid lens);

[0064] S5: performing trajectory tracking and prediction on all the latest monitored objects (i.e., the monitored objects in the latest historical monitoring video) in the latest historical monitoring video acquired by the liquid lens (i.e., the latest historical monitoring video of a preset duration) (i.e., performing trajectory tracking on the latest monitored objects, and further predicting their subsequent movement trajectories based on the tracking trajectories determined by the trajectory tracking, wherein the trajectory tracking can be based on a simple MATLAB method for finding the area where the monitored objects are located from the historical monitoring video frames of the latest acquired historical monitoring video and performing video tracking, i.e., obtaining the centroid (i.e., the geometric center) of the area where the monitored objects are located to locate the monitored objects, and finally connecting the target positions of the monitored objects in each of the found historical monitoring video frames to obtain the tracking trajectory), and determining the predicted movement trajectories of all the latest monitored objects in the monitoring video frames (i.e., the trajectories that the latest monitored objects may pass through in the subsequent process after performing trajectory tracking and prediction on the latest historical monitoring video);

[0065] S6: Predicting a focus position movement trajectory within the rough focus range based on the predicted movement trajectory (i.e., a predicted movement trajectory of a suitable focus position of the liquid lens in the subsequent process, so that the display effect of the area where the monitored object is located in the monitoring video frames subsequently acquired by the liquid lens meets the requirements);

[0066] S7: Pre-focus control is performed on the liquid lens based on the focus position movement trajectory (i.e., the focus position of the liquid lens is pre-set and controlled according to the focus position in the next adjacent video frame in the focus position movement trajectory), and a focus control result of the liquid lens is obtained (i.e., the result obtained after the pre-focus control of the liquid lens).

[0067] Based on the concentrated distribution area of the monitored object in the historical monitoring video, the rough focus range of the liquid lens is preliminarily determined. Then, by tracking and predicting the trajectory of the monitored object in the monitoring video most recently acquired by the liquid lens, the next appropriate focus position movement trajectory of the liquid lens is further predicted in detail. Based on the intersection of the ranges defined by the two aforementioned focus range and position determinations, a focus position movement trajectory with high prediction accuracy and high rationality is determined, and pre-focus control of the liquid lens is implemented based on the predicted focus position movement trajectory, which greatly ensures the display effect of the monitored object in the monitoring video frames subsequently acquired by the liquid lens.

[0068] Example 3:

[0069] Based on Example 1, the image data analysis and processing method based on the liquid lens includes: S1: based on the massive amount of historical surveillance video acquired by the liquid lens, analyzing the concentrated distribution area of the surveillance objects in the surveillance scene of the liquid lens in the surveillance video frame area, including:

[0070] Determine, in each historical surveillance video frame of the massive amount of historical surveillance video acquired by the liquid lens, the area where the monitored object is located in the monitoring scene of the liquid lens, and use it as the monitoring area (i.e., the area where the monitored object is located in the historical surveillance video frame);

[0071] The ratio of the number of times each pixel position in the surveillance video frame area exists in all surveillance areas to the total number of historical surveillance video frames is regarded as the frequency of the surveillance object at the corresponding pixel position (that is, the frequency ratio of the surveillance object covering the corresponding pixel position in all surveillance areas in all historical surveillance video frames);

[0072] The pixel positions in the monitoring video frame area whose existence frequency is not less than the concentrated frequency threshold (that is, the preset minimum threshold that the pixel position needs to meet when it is determined to be a concentrated distribution position) are regarded as concentrated distribution positions (that is, the pixel positions whose existence frequency is not less than the concentrated frequency threshold), and all concentrated distribution positions are summarized to obtain the concentrated distribution area of the monitoring object in the monitoring scene of the liquid lens in the monitoring video frame area.

[0073] The above process accurately determines the area where the monitored objects are concentrated in the monitoring video frame area by comparing the frequency ratio of each pixel position in the historical monitoring video frames covered by all monitoring areas of the monitored objects in all historical monitoring video frames with the concentration frequency threshold.

[0074] Example 4:

[0075] Based on Example 1, the image data analysis and processing method based on the liquid lens, S2: calculating the defocus data of the monitored object in each historical monitoring video frame, including:

[0076] Based on pixel gradient information in the historical surveillance video frame (i.e., information including pixel gradient value data in different directions in the historical surveillance video frame), determine the focus pixel position in the historical surveillance video frame (i.e., the focus position formed in the historical surveillance video frame when the liquid lens captured a single historical surveillance video frame);

[0077] The distance between each pixel position and the focus pixel position in the monitoring area corresponding to the monitoring object in each historical monitoring video frame is regarded as the defocus data of the monitoring object in the historical monitoring video frame.

[0078] Based on the image gradient information of the historical surveillance video frames, the focus pixel position formed by the liquid lens in the corresponding historical surveillance video frame when acquiring each historical surveillance video frame is accurately determined, and the distance between the monitoring area where the monitored object is located and the focus pixel position is used as the defocus data, that is, the deviation data of the position of the monitored object in the historical surveillance video frame relative to the focus position in the historical surveillance video frame is accurately determined.

[0079] Example 5:

[0080] Based on Example 4, the image data analysis and processing method based on the liquid lens determines the focus pixel position in the historical monitoring video frame based on the pixel gradient information in the historical monitoring video frame, including:

[0081] Determine all pixel point sequences in each preset direction (e.g., the positive and negative directions of the horizontal and vertical axes in a preset coordinate system) of the historical surveillance video frame (i.e., multiple pixel points formed by sorting the pixels in the historical surveillance video frame according to a single preset direction, such as multiple rows of pixel sequences and multiple columns of pixel sequences), and generate a pixel gradient sequence for all pixel point sequences in each preset direction based on the differences in pixel values of adjacent pixels in the pixel point sequence in the historical surveillance video frame (i.e., a sequence formed by sorting the pixel value differences between adjacent pixels in the pixel point sequence according to the sorting order in the pixel point sequence);

[0082] Determine a partial focus range dividing boundary between adjacent pixel points corresponding to the maximum pixel value difference in all pixel gradient sequences (i.e., the boundary between adjacent pixel points corresponding to the maximum pixel value difference in the pixel gradient sequence is also a partial focus range boundary);

[0083] All the partial focus range boundaries are connected to enclose the focus range of the historical monitoring video frame (i.e., the area enclosed by the connection of all the partial focus range boundaries), and the center position of the focus range is used as the focus pixel position of the historical monitoring video frame.

[0084] Based on the maximum value in a pixel gradient sequence representing the change in the pixel value difference between adjacent pixels in a pixel point sequence in multiple preset directions of historical surveillance video frames, the boundaries between pixels with higher local sharpness are determined as partial boundaries of the focusing range, and all partial boundaries are connected to obtain the focusing range. The geometric center of the focusing range is further taken as the focusing pixel position, thereby accurately determining the focusing pixel position formed in the historical surveillance video frame when the liquid lens acquires the historical surveillance video frame.

[0085] Example 6:

[0086] Based on Example 5, the image data analysis and processing method based on the liquid lens, S3: performing partitioned and focused restoration of the historical surveillance video frames based on the defocus data to obtain the restored historical surveillance video frames, including:

[0087] The ratio of the distance between each pixel position and the focused pixel position in the monitoring area in the historical monitoring video frame in the out-of-focus data to the maximum value of the distances between all pixel positions and the focused pixel position in the monitoring area is used as the first repair weight of the corresponding pixel in the monitoring area (that is, the value representing the degree of repair when the pixel position at the pixel position in the monitoring area is repaired based on the distance between the pixel position and the focused pixel position. The larger the first repair weight, the better the display effect that needs to be restored (that is, the larger the clarity, local sharpness or contrast values are));

[0088] Determine the center position of the monitoring area (i.e., the geometric center position of the monitoring area), and use the ratio of the distance between each pixel position in the monitoring area and the center position of the monitoring area to the maximum value of the distances between all pixel positions in the monitoring area and the center position of the monitoring area as the second restoration weight of the corresponding pixel in the monitoring area (i.e., a value that represents the degree of restoration of the pixel position when the distance between the pixel position in the monitoring area and the center position of the monitoring area is restored. The smaller the second restoration weight, the better the display effect to be restored (i.e., the larger the clarity, local sharpness, or contrast values).

[0089] The difference between the first repair weight and the second repair weight of the pixel in the monitoring area is used as the final repair weight of the corresponding pixel (that is, the repair weight determined by the weight influence of the distance between the pixel position in the comprehensive monitoring area and the focus pixel position and the distance between the pixel position and the center position of the monitoring area when repairing the pixel at the pixel position. The final repair weight is mainly determined based on the principle that the farther the pixel position is from the focus pixel position, the greater the degree of repair required, and the farther the pixel position is from the monitoring center position, the less the degree of repair required);

[0090] The historical surveillance video frames with the final repair weights of all pixels in the monitoring area marked are input into the video frame partition key repair model (the video frame partition key repair model is a model trained in advance using a large number of surveillance video frames with the final repair weights of each pixel in the monitoring area marked and the repaired surveillance video frames that meet the display requirements after the corresponding surveillance video frames are repaired as training samples. The video frame partition key repair model can generate corresponding repaired surveillance video frames that meet the display requirements based on the input historical surveillance video frames with the final repair weights of all pixels in the monitoring area marked. Compared with the historical surveillance video frames before repair, the repaired surveillance video frames generated by the repair model only focus on repairing the monitoring area, that is, realizing the focused repair of the monitoring area of the input surveillance video frames), and obtain the repaired historical surveillance video frames (that is, the repaired historical surveillance video frames generated by the video frame partition key repair model).

[0091] The above process determines the final repair weight of each pixel in the monitoring area based on the principle that the farther away from the focus pixel position, the greater the degree of repair required, and the farther away from the monitoring center position, the smaller the degree of repair required. The historical monitoring video frames that mark the final repair weights of all pixels in the monitoring area are used as model input samples, and the pre-prepared video frame partition key repair model is input to realize the key partition repair of the historical monitoring video frames.

[0092] Example 7:

[0093] Based on Example 2, the image data analysis and processing method based on the liquid lens, S4: analyzing the rough focus range of the liquid lens based on the concentrated distribution area, includes:

[0094] An effective focus radius based on preset requirements (i.e., the maximum allowable distance between a pixel position in a video frame and the focus position of the liquid lens in the video frame, as defined by the original camera device parameters of the liquid lens, for the pixel display effect at that pixel position in the video frame to meet the preset display requirements);

[0095] Determine the tangent line of the concentrated distribution area at each contour point of the concentrated distribution area, and use the straight line passing through the corresponding contour point and perpendicular to the corresponding tangent line as the assumed inner diameter straight line of the concentrated distribution area at the corresponding contour point;

[0096] Assuming that the distance between the inner diameter straight line and the corresponding contour point is the effective focusing radius, the position located inside the concentrated distribution area is regarded as the outermost focus center position of the corresponding contour point (that is, the contour point of the corresponding rough focus range determined by the corresponding contour point within the concentrated distribution area);

[0097] The area enclosed by connecting the outermost focus center positions of all contour points in the concentrated distribution area is regarded as the rough focus range of the liquid lens.

[0098] The above process uses the area determined by uniformly reducing the effective focusing radius from the concentrated distribution area inward as the coarse focusing range of the liquid lens. This ensures that the monitoring area active within the coarse focusing range can meet the preset display requirements corresponding to the effective focusing radius, that is, determines the effective value range of the focus pixel position that can ensure the display effect of the monitored object.

[0099] Example 8:

[0100] Based on Example 2, the image data analysis and processing method based on the liquid lens, S5: tracking and predicting the trajectories of all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens, and determining the predicted movement trajectories of all the latest monitored objects in the monitoring video frame, including:

[0101] Track all the latest monitored objects in the latest historical monitoring video acquired by the liquid lens to obtain the current tracking trajectory of all the latest monitored objects (i.e., the movement trajectory of the latest monitored objects in the latest historical monitoring video);

[0102] Based on the current tracking trajectory and the category of the latest monitored object (such as pedestrians, vehicles, traffic signs) and a preset trajectory prediction model (that is, a model pre-trained using partial tracking trajectories of a large number of monitored objects of different categories and all tracking trajectories belonging to the partial tracking trajectories as training samples, the preset trajectory prediction model can use the input monitored object category and the corresponding partial tracking trajectory to predict the corresponding predicted movement trajectory), the predicted movement trajectory of all the latest monitored objects in the monitored video frame is determined.

[0103] The above process tracks the trajectory of the latest monitored object in the historical monitoring video captured by the liquid lens, and based on the current tracking trajectory determined by tracking and the category of the latest monitored object and the preset trajectory prediction model, predicts the predicted movement trajectory of the latest monitored object in the monitoring video frame, that is, achieves an accurate prediction of the next movement trajectory of the monitored object.

[0104] Example 9:

[0105] Based on Example 2, the image data analysis and processing method based on the liquid lens, S6: predicting the focus position movement trajectory within the rough focus range based on the predicted movement trajectory, includes:

[0106] The predicted movement trajectories of all the latest monitored objects in the latest historical surveillance video are averaged in the same video frame (i.e., the coordinate values corresponding to the predicted movement trajectories of all the latest monitored objects in the same video frame are averaged), and the focus position of the corresponding video frame is obtained (i.e., the position corresponding to the coordinate values obtained after averaging);

[0107] The focus positions of all video frames are sorted and fitted according to the video frame order to obtain the initial focus position movement trajectory (that is, the initial focus position movement trajectory obtained by averaging and fitting the predicted movement trajectory of all the latest monitored objects in the latest historical monitoring video);

[0108] The portion of the initial focus position movement trajectory outside the rough focus range is deleted to obtain the focus position movement trajectory.

[0109] The above process uses the rough focus range determined in the previous steps to further specifically denoise the initial focus position movement trajectory obtained by averaging and fitting the predicted movement trajectories of all the latest monitored objects, thereby achieving a reasonable prediction of the focus position of the liquid lens in the next video frame, thereby ensuring that the display effect of the video frame acquired by the liquid lens meets the requirements.

[0110] Example 10:

[0111] The present invention proposes an image data analysis and processing system based on a liquid lens, referring to Figure 3 ,include:

[0112] The concentrated area analysis module is used to analyze the concentrated distribution area of the monitored objects in the monitoring scene of the liquid lens in the monitoring video frame area based on the massive historical monitoring videos acquired by the liquid lens;

[0113] A defocus data calculation module is used to calculate the defocus data of the monitored object in each historical monitoring video frame;

[0114] The video frame repair module is used to perform partition-focused repair on historical surveillance video frames based on defocus data to obtain repaired historical surveillance video frames.

[0115] Based on the concentrated distribution area of the monitored object and the defocus degree of the monitored area corresponding to the monitored object, the video frames obtained based on the liquid lens are partitioned and focused on repair, which not only ensures that the repaired image can meet the display requirements but also improves the repair efficiency.

[0116] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. An image data analysis and processing method based on a liquid lens, characterized in that: include: S1: Based on the massive amount of historical surveillance videos acquired by the liquid lens, the concentrated distribution area of the surveillance objects in the surveillance scene of the liquid lens is analyzed in the surveillance video frame area; S2: Calculate the defocus data of the monitored object in each historical monitoring video frame; S3: Performing partition-based key restoration on historical surveillance video frames based on the defocus data to obtain restored historical surveillance video frames, including: The ratio between the distance between each pixel position and the focused pixel position in the monitoring area in the historical monitoring video frame in the out-of-focus data and the maximum value of the distances between all pixel positions and the focused pixel position in the monitoring area is used as the first repair weight of the corresponding pixel in the monitoring area; Determine the center position of the monitoring area, and use the ratio of the distance between each pixel position in the monitoring area and the center position of the monitoring area to the maximum value of the distances between all pixel positions in the monitoring area and the center position of the monitoring area as the second restoration weight of the corresponding pixel in the monitoring area; The difference between the first restoration weight and the second restoration weight of the pixel in the monitoring area is regarded as the final restoration weight of the corresponding pixel; The historical monitoring video frames with the final restoration weights of all pixels in the monitoring area marked are input into the video frame partition key restoration model to obtain the restored historical monitoring video frames.

2. The image data analysis and processing method based on liquid lens according to claim 1, characterized in that: Also includes: S4: Analyze the rough focus range of the liquid lens based on the concentrated distribution area; S5: Tracking and predicting the trajectories of all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens, and determining the predicted movement trajectories of all the latest monitored objects in the monitoring video frames; S6: Predicting a focus position movement trajectory within a rough focus range based on the predicted movement trajectory; S7: Pre-focusing the liquid lens based on the focus position movement trajectory to obtain a focus control result of the liquid lens.

3. The image data analysis and processing method based on liquid lens according to claim 1, characterized in that: S1: Based on the massive amount of historical surveillance videos acquired by the liquid lens, the concentrated distribution areas of the surveillance objects in the surveillance scenes of the liquid lens are analyzed in the surveillance video frame area, including: Determine the area where the monitored object is located in the monitoring scene of the liquid lens in each historical monitoring video frame of the massive historical monitoring video acquired by the liquid lens, and use the area as the monitoring area; The ratio of the number of times each pixel position in the monitoring video frame area exists in all monitoring areas to the total number of historical monitoring video frames is regarded as the frequency of the monitoring object at the corresponding pixel position; The pixel positions in the monitoring video frame area with a frequency not less than the concentration frequency threshold are regarded as concentrated distribution positions, and all concentrated distribution positions are summarized to obtain the concentrated distribution area of the monitoring object in the monitoring scene of the liquid lens in the monitoring video frame area.

4. The method for analyzing and processing image data based on a liquid lens according to claim 1, wherein: S2: Calculate the defocus data of the monitored object in each historical monitoring video frame, including: Determine the focus pixel position in the historical surveillance video frame based on pixel gradient information in the historical surveillance video frame; The distance between each pixel position and the focus pixel position in the monitoring area corresponding to the monitoring object in each historical monitoring video frame is regarded as the defocus data of the monitoring object in the historical monitoring video frame.

5. The image data analysis and processing method based on liquid lens according to claim 4, characterized in that: Determine the focus pixel position in the historical surveillance video frame based on pixel gradient information in the historical surveillance video frame, including: Determine all pixel point sequences in each preset direction of the historical surveillance video frame, and generate a pixel gradient sequence for all pixel point sequences in each preset direction based on the difference in pixel values of adjacent pixel points in the pixel point sequence in the historical surveillance video frame; Determine a partial focus range dividing boundary between adjacent pixel points corresponding to the maximum pixel value difference in all pixel gradient sequences; The boundaries of all partial focus ranges are connected to form the focus range of the historical monitoring video frame, and the center position of the focus range is used as the focus pixel position of the historical monitoring video frame.

6. The method for analyzing and processing image data based on a liquid lens according to claim 2, wherein: S4: Analyze the rough focus range of the liquid lens based on the concentrated distribution area, including: Effective focusing radius based on preset requirements; Determine the tangent line of the concentrated distribution area at each contour point of the concentrated distribution area, and use the straight line passing through the corresponding contour point and perpendicular to the corresponding tangent line as the assumed inner diameter straight line of the concentrated distribution area at the corresponding contour point; Assuming that the distance between the inner diameter straight line and the corresponding contour point is the effective focusing radius, the position located inside the concentrated distribution area is regarded as the outermost focusing center position of the corresponding contour point; The area enclosed by connecting the outermost focus center positions of all contour points in the concentrated distribution area is regarded as the rough focus range of the liquid lens.

7. The method for analyzing and processing image data based on a liquid lens according to claim 2, wherein: S5: Track and predict the trajectories of all the latest monitored objects in the historical surveillance video recently acquired by the liquid lens, and determine the predicted movement trajectories of all the latest monitored objects in the surveillance video frames, including: Track all the latest monitored objects in the historical monitoring video recently acquired by the liquid lens to obtain the current tracking trajectories of all the latest monitored objects; Based on the current tracking trajectory, the category of the latest monitored object and the preset trajectory prediction model, the predicted movement trajectory of all the latest monitored objects in the monitored video frame is determined.

8. The method for analyzing and processing image data based on a liquid lens according to claim 2, wherein: S6: Predicting a focus position movement trajectory within a rough focus range based on the predicted movement trajectory, including: The predicted movement trajectories of all the latest monitored objects in the latest historical monitoring video are averaged in the same video frame to obtain the focus position of the corresponding video frame; The focus positions of all video frames are sorted and fitted according to the video frame order to obtain the initial focus position movement trajectory; The portion of the initial focus position movement trajectory outside the rough focus range is deleted to obtain the focus position movement trajectory.

9. An image data analysis and processing system based on a liquid lens, characterized in that: include: The concentrated area analysis module is used to analyze the concentrated distribution area of the monitored objects in the monitoring scene of the liquid lens in the monitoring video frame area based on the massive historical monitoring videos acquired by the liquid lens; A defocus data calculation module is used to calculate the defocus data of the monitored object in each historical monitoring video frame; The video frame repair module is used to perform partition-based repair of historical surveillance video frames based on defocus data to obtain repaired historical surveillance video frames, including: The ratio between the distance between each pixel position and the focused pixel position in the monitoring area in the historical monitoring video frame in the out-of-focus data and the maximum value of the distances between all pixel positions and the focused pixel position in the monitoring area is used as the first repair weight of the corresponding pixel in the monitoring area; Determine the center position of the monitoring area, and use the ratio of the distance between each pixel position in the monitoring area and the center position of the monitoring area to the maximum value of the distances between all pixel positions in the monitoring area and the center position of the monitoring area as the second restoration weight of the corresponding pixel in the monitoring area; The difference between the first restoration weight and the second restoration weight of the pixel in the monitoring area is regarded as the final restoration weight of the corresponding pixel; The historical monitoring video frames with the final restoration weights of all pixels in the monitoring area marked are input into the video frame partition key restoration model to obtain the restored historical monitoring video frames.

Images