Image processing method and system based on deep learning

Abstract

This invention discloses an image processing method and system based on deep learning. The method includes: receiving a video file transmitted from an external terminal and performing a security check on the external terminal; separating image frames from the video file and extracting important image frames containing target features from the image frames; correcting the important image frames, adjusting the sharpness of the corrected important image frames, and outputting a high-resolution image; constructing a target feature pattern model based on deep learning, and inputting the obtained high-resolution image into the target feature pattern model to determine the target feature pattern. Using the technical solution of this invention can improve image processing efficiency and quickly determine target feature patterns.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and system based on deep learning. Background Technology

[0002] Image processing technology is the technique of using computers to process image information. It mainly includes image digitization, image enhancement and restoration, image data encoding, image segmentation, and image recognition.

[0003] When external devices request a server to track target trajectory patterns, they upload a large number of video files to the server, while the server receives a large number of video files uploaded by external devices every day. Within a video, the target of interest is usually traceable, allowing for the prediction of its subsequent trajectory patterns. However, existing image processing methods typically involve extensive manual screening for trajectory rule determination, resulting in low efficiency. Therefore, this invention proposes a deep learning-based image processing method and system that enables rapid image processing to improve the screening speed of target trajectory rules. Summary of the Invention

[0004] This invention provides a deep learning-based image processing method, comprising:

[0005] Receive video files transmitted from external terminals and perform security checks on the external terminals;

[0006] Separate image frames from the video file and extract important image frames containing target features from the image frames;

[0007] The system corrects important image frames, adjusts the sharpness of corrected important image frames, and outputs high-resolution images.

[0008] A target feature pattern model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature pattern model to determine the target feature pattern.

[0009] As described above, a deep learning-based image processing method extracts external terminal information from a video file, calculates the security level of the external terminal, and if the security level of the external device is higher than that of the server, image processing of the video file of the external device is allowed; otherwise, the external terminal request is rejected.

[0010] The deep learning-based image processing method described above includes the following sub-steps for extracting important image frames containing target features from image frames:

[0011] From all the image frames separated from the video, find the first important image frame containing the target features;

[0012] The target features in the first important image frame are processed into grayscale to determine the grayscale value of the target features;

[0013] The frame distance between the image frame to be extracted and the first important image frame is calculated based on the grayscale distribution value of the target feature. All important image frames containing the target feature are then determined based on the frame distance.

[0014] The deep learning-based image processing method described above includes correcting important image frames, specifically comprising the following sub-steps:

[0015] Extract the target region from all important image frames, back-project the target region, compare the original image and the back-projected image to see if there are any differences, and select several important image frames whose differences are within a predetermined range.

[0016] Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames with sharpness lower than the preset value;

[0017] Adjust the pixels, brightness, and feature regions of other important image frames according to the reference image frame.

[0018] The image processing method based on deep learning described above includes extracting target regions from important image frames. Specifically, this involves: obtaining foreground and background regions and unknown regions in the important image frames; in order to make the points in the unknown regions as close as possible to the foreground and background regions, taking each point in the unknown region as the center, obtaining the distance between the pixel color in its radius and the pixel color of that point; using the pixel distance greater than a set maximum threshold as the foreground region and the pixel distance less than a set minimum threshold as the background region, thereby narrowing the range of the unknown regions.

[0019] The present invention also provides a deep learning-based image processing system, comprising:

[0020] The security check module is used to receive video files transmitted from external terminals and perform security checks on the external terminals.

[0021] The important image frame determination module is used to separate image frames from video files and extract important image frames containing target features from the image frames;

[0022] The high-resolution image output module is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images.

[0023] The deep learning module is used to build a target feature pattern model based on deep learning, and inputs the obtained high-resolution image into the target feature pattern model to determine the target feature pattern.

[0024] In the deep learning-based image processing system described above, the security check module is specifically used to extract external terminal information from the video file, calculate the security level of the external terminal, and if the security level of the external device is higher than that of the server, image processing of the video file of the external device is allowed; otherwise, the external terminal request is rejected.

[0025] As described above, in a deep learning-based image processing system, the important image frame determination module is specifically used to find the first important image frame containing the target feature from all image frames separated from the video; perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature; calculate the frame distance between the image frame to be extracted and the first important image frame based on the grayscale distribution value of the target feature; and determine all important image frames containing the target feature based on the frame distance.

[0026] As described above, a deep learning-based image processing system includes a high-resolution image output module, specifically used to extract target regions from all important image frames, back-project the target regions, compare the original image and the back-projected image to determine if there are differences, select several important image frames whose differences are within a predetermined range, calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames whose sharpness is lower than a preset value; and adjust the pixels, brightness, and feature regions of other important image frames according to the reference image frames.

[0027] In the deep learning-based image processing system described above, the high-resolution image output module extracts target regions from important image frames. Specifically, it obtains foreground and background regions and unknown regions in the important image frames. To make the points in the unknown regions as close as possible to the foreground and background regions, it takes each point in the unknown region as the center, obtains the distance between the pixel color in its radius and the pixel color of that point, and takes the pixel distance greater than a set maximum threshold as the foreground region and the pixel distance less than a set minimum threshold as the background region, thereby narrowing the range of the unknown regions.

[0028] The beneficial effects achieved by this invention are as follows: By adopting the technical solution of this invention, image processing efficiency can be improved and target feature patterns can be quickly determined. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0030] Figure 1This is a flowchart of an image processing method based on deep learning provided in Embodiment 1 of the present invention;

[0031] Figure 2 This is a schematic diagram of an image processing system based on deep learning provided in Embodiment 2 of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example 1

[0034] like Figure 1 As shown, this embodiment of the invention provides an image processing method based on deep learning, including:

[0035] Step S110: The server receives the video file transmitted from the external terminal and performs a security check on the external terminal.

[0036] In this embodiment, external terminal information is extracted from the video file, and the security level of the external terminal is calculated. If the security level of the external device is higher than that of the server, image processing of the video file of the external device is allowed; otherwise, the external terminal request is rejected.

[0037] Specifically, the security check of external terminals includes the following sub-steps:

[0038] Step 111: Extract external terminal information from the video file of the external terminal, including external terminal device information, data packet sending and receiving time, IP address, port, etc.

[0039] Step 112: Determine whether the external terminal's IP address and port are allowed to be directly accessed by the server. If so, allow image processing of the video files of the external device; otherwise, proceed to step 113.

[0040] If the IP address and port of the external device are stored as secure addresses on the server, subsequent image processing of the video files from the external device is directly permitted. If the IP address and port of the external device are not stored on the server, the external device may pose a security risk, and therefore security authentication of the external device is required.

[0041] Step 113: Calculate the security level of the external device and the server's access level to the external device. If the security level of the external device is greater than the server's access level, then image processing of the video file on the external device is allowed; otherwise, the external terminal request is rejected.

[0042] Specifically, using the formula The security level of an external device is calculated, where Se represents the security level of the external device; λ1 is the weight of the impact of the external device risk given by the firewall on the security level; λ2 is the weight of the impact of the relationship between the external device access address and the firewall filtering rules on the security assessment value; e = 2.718; μ is the risk factor of the firewall blocking the external device's IP port; Nr is the number of vulnerabilities in the firewall that affect the data processing of the external device's IP port; and Ns is the total number of vulnerabilities in the firewall. A vector representation of the external device's IP address; t is a vector representation of the firewall filtering rule, where t ranges from 1 to T, and T is the total number of characters in the firewall power rule.

[0043] Then use the formula Calculate the server's access openness to external devices, where Kf represents the server's access openness to external devices; e = 2.718; N1 represents the total number of external device service categories that belong to the server category; N2 represents the total number of external device service categories that do not belong to the server category; β IP This indicates whether the external device's IP address is an invalid IP domain of the server. If not, then β... IP If β is 1, then β IP =0; This indicates whether the server and external devices share a common security protocol. If so, then... otherwise

[0044] If Se≥Kf, it indicates that the external device has a high level of security, and image processing of the video file on the external device is allowed; otherwise, it indicates that the external device has a low level of security, and the server rejects the external terminal's request.

[0045] Step S120: Separate image frames from the video and extract important image frames containing target features from the image frames;

[0046] Specifically, extracting important image frames containing target features from image frames includes the following sub-steps:

[0047] Step 121: From all the image frames separated from the video, find the first important image frame containing the target features;

[0048] Step 122: Perform grayscale processing on the target features in the first important image frame to determine the grayscale value of the target features;

[0049] Specifically, the RGB values ​​of each pixel are extracted from the target feature image region in the first important image frame. Then, the grayscale value of each pixel in the target feature is calculated according to the formula Gray(i,j)=WR*R(i,j)+WG*G(i,j)+WB*B(i,j), where WR, WG, and WB are the weights of the pixel's RGB values, respectively. The grayscale value of the target feature in the first important image frame, i.e.,

[0050] Step 123: Calculate the frame distance between the image frame to be extracted and the first important image frame based on the grayscale distribution value of the target feature, and determine all important image frames containing the target feature based on the frame distance;

[0051] Specifically, after determining the first important image frame, the frame distance between subsequent image frames and the first important image frame is calculated, i.e. Where N is the number of image frames. This represents the grayscale value of the (i+1)th and i-th image frames. This represents the grayscale value of the first important image frame. Specifically, determining all important image frames containing target features based on frame distance involves sorting all frames according to their frame distance and selecting several images with the highest frame distance, or several images with a frame distance higher than a preset threshold, as important image frames.

[0052] Step S130: Correct important image frames, adjust the sharpness of corrected important image frames, and output high-resolution images;

[0053] In this embodiment of the application, the correction of important image frames specifically includes the following sub-steps:

[0054] Step 131: Extract the target region from all important image frames, back-project the target region, compare the original image and the back-projected image to see if there are any differences, and select several important image frames whose differences are within a predetermined range.

[0055] The process of extracting the target region from all important image frames specifically includes: obtaining the foreground and background regions and the unknown region Ii = αFi + (1-α)Bi from the important image frames, where α represents transparency, Fi is the foreground pixel, Bi is the background pixel, and i represents the i-th pixel. To make the points in the unknown region as close as possible to the foreground and background regions, the distance between the pixel color within the radius of each point in the unknown region and the pixel color of that point is obtained. Pixels with a distance greater than a set maximum threshold are designated as foreground regions, while pixels with a distance less than a set minimum threshold are designated as background regions, thus narrowing down the range of unknown areas. The values ​​obtained from back projection are compared with those obtained from orthographic projection. If the results are outside a predetermined range, it indicates that the image is very blurry, and the image is discarded.

[0056] Step 132: Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames with sharpness lower than the preset value;

[0057] Specifically, using the formula Calculate the sharpness of each important image frame, where x is the number of pixels in length of the important image frame, y is the number of pixels in width of the important image frame, and z is the image size. The image with the highest sharpness is used as the reference image frame.

[0058] If any important image frames have a resolution lower than a preset value, then this data is adjusted for resolution. Specifically, this involves: identifying blurred regions within these important image frames with resolution below the preset value, reconstructing these blurred regions, and encoding the reconstructed image frames. Reconstructing the blurred regions specifically involves pixel reconstruction of these regions, reconstructing the pixel coordinates. n represents a point on a circle with the center of the starting region as the origin and the diagonal length of the starting region as the radius. Typically, n = 4 points are chosen as four points evenly distributed on the circle. d i R is the distance between the reconstructed coordinates of the selected starting region block and the circumferential pixel coordinates. j R1 represents the pixel value of the pixel coordinates of the selected point on the circumference. Specifically, d1 is the distance between the reconstructed coordinates of the selected starting region block and the pixel coordinates of the adjacent region block to the left, and R1 represents the pixel value of the pixel coordinates of the adjacent region block to the right; d2 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block to the right, and R2 represents the pixel value of the pixel coordinates of the adjacent region block to the left; d3 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block above, and R3 represents the pixel value of the pixel coordinates of the adjacent region block below; d4 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block below, and R4 represents the pixel value of the pixel coordinates of the adjacent region block above.

[0059] Step 133: Adjust the pixels, brightness, and feature regions of other important image frames according to the reference image frame;

[0060] Specifically, pixel adjustment includes: determining the pixel values ​​of the target region in other important image frames based on the back-projected images of those frames. Where n represents the number of projected lines in the image, q k,iThis represents the i-th projection line passing through pixel k. The pixel ratio Z = Pv / h(k) between these target regions and the target region of the reference image frame is calculated. V Using the reference image frame's pixel values ​​as a reference, pixel scaling is applied to other important image frames based on their pixel ratios. Brightness / contrast adjustment specifically includes: calculating the brightness / contrast of the reference image frame as R*0.299 + G*0.587 + B*0.114, with the weights in the brightness / contrast calculation adjusted as needed; and then adjusting the brightness / contrast of other key image frames according to the reference image frame's brightness / contrast. Feature region adjustment specifically includes: performing rotation and proportional scaling on other key image frames based on the reference image frame.

[0061] Step S140: Construct a target feature pattern model based on deep learning, and input the obtained high-resolution image into the target feature pattern model to determine the target feature pattern;

[0062] Specifically, a target feature pattern model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature pattern model to determine the target feature pattern. This includes the following sub-steps:

[0063] Step 141: Extract target features from the obtained high-resolution image to form a target feature vector;

[0064] Step 142: Input the target feature vector set into the target feature pattern model, train multiple damping trend prediction models to obtain different sub-trend prediction models, use each sub-trend prediction model to predict the trend of the target feature vector set, and estimate the set of weights of each sub-prediction model through the trend prediction results.

[0065] Specifically, the target pedestrian feature vector set is input into the target pedestrian feature trend model, and multiple damping trend prediction models are trained using the target pedestrian feature vector set, with each damping trend prediction model serving as a sub-prediction model. The sub-prediction model is then used to predict the feature vector set of the target pedestrian, and the prediction result is obtained. The prediction result is then used with a formula. The set of weights for estimating the sub-prediction model {λ1, λ2, λ3, ..., λ4} is given. T}, where x i y represents the target shape feature in the target feature vector. i Let μ1 and μ2 be the target spatial relationship features of the target feature vector, μ1 and μ2 be the influence weights of the target shape features and the target spatial relationship features, n be the total number of target feature vectors, and T be the number of sub-prediction models.

[0066] Step 143: Find the optimal value corresponding to each weight in the set of weights, and determine the target feature pattern by combining the optimal values ​​of each sub-prediction model and its corresponding weight.

[0067] Specifically, calculate each sub-prediction model The set of weights {λ1, λ2, λ3, ..., λ T In the context of prediction models, the optimal value corresponds to each weight; through each sub-prediction model... The optimal values ​​of their corresponding weights {λ1, λ2, λ3, ..., λ} T Combining to determine the target characteristics and patterns

[0068] Example 2

[0069] like Figure 2 As shown, this embodiment of the invention provides a deep learning-based image processing system 2, comprising:

[0070] Security check module 21 is used to receive video files transmitted from external terminals and perform security checks on the external terminals;

[0071] The security check module 21 is used to extract external terminal information from the video file and calculate the security level of the external terminal. If the security level of the external device is higher than that of the server, image processing of the video file of the external device is allowed; otherwise, the external terminal request is rejected. Specifically, the security check module 21 is used to extract external terminal information from the video file of the external terminal, including external terminal device information, data packet sending and receiving time, IP address, port, etc.; determine whether the external terminal's IP address and port are allowed to be directly accessed by the server. If so, image processing of the video file of the external device is allowed; otherwise, the security level of the external device is calculated, and the server's access openness to the external device is calculated. If the security level of the external device is greater than that of the server, image processing of the video file of the external device is allowed; otherwise, the external terminal request is rejected.

[0072] Specifically, using the formula The security level of an external device is calculated, where Se represents the security level of the external device; λ1 is the weight of the impact of the external device risk given by the firewall on the security level; λ2 is the weight of the impact of the relationship between the external device access address and the firewall filtering rules on the security assessment value; e = 2.718; μ is the risk factor of the firewall blocking the external device's IP port; Nr is the number of vulnerabilities in the firewall that affect the data processing of the external device's IP port; and Ns is the total number of vulnerabilities in the firewall. A vector representation of the external device's IP address; t is a vector representation of the firewall filtering rule, where t ranges from 1 to T, and T is the total number of characters in the firewall power rule.

[0073] Then use the formula Calculate the server's access openness to external devices, where Kf represents the server's access openness to external devices; e = 2.718; N1 represents the total number of external device service categories that belong to the server category; N2 represents the total number of external device service categories that do not belong to the server category; β IP This indicates whether the external device's IP address is an invalid IP domain of the server. If not, then β... IP If β is 1, then β IP =0; This indicates whether the server and external devices share a common security protocol. If so, then... otherwise

[0074] If Se≥Kf, it indicates that the external device has a high level of security, and image processing of the video file on the external device is allowed; otherwise, it indicates that the external device has a low level of security, and the server rejects the external terminal's request.

[0075] Important image frame determination module 22 is used to separate image frames from the video file and extract important image frames containing target features from the image frames;

[0076] The important image frame determination module 22 is specifically used to find the first important image frame containing the target feature from all the image frames separated from the video; to perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature; to calculate the frame distance between the image frame to be extracted and the first important image frame based on the grayscale distribution value of the target feature; and to determine all important image frames containing the target feature based on the frame distance.

[0077] Specifically, the RGB values ​​of each pixel are extracted from the target feature image region in the first important image frame. Then, the grayscale value of each pixel in the target feature is calculated according to the formula Gray(i,j)=WR*R(i,j)+WG*G(i,j)+WB*B(i,j), where WR, WG, and WB are the weights of the pixel's RGB values, respectively. The grayscale value of the target feature in the first important image frame, i.e., After identifying the first important image frame, the frame interval between subsequent image frames and the first important image frame is calculated. Where N is the number of image frames. This represents the grayscale value of the (i+1)th and i-th image frames. This represents the grayscale value of the first important image frame. Specifically, determining all important image frames containing target features based on frame distance involves sorting all frames according to their frame distance and selecting several images with the highest frame distance, or several images with a frame distance higher than a preset threshold, as important image frames.

[0078] The high-resolution image output module 23 is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images.

[0079] The high-resolution image output module 23 is specifically used to extract the target region from all important image frames, back-project the target region, compare whether there is a difference between the original image and the back-projected image, select several important image frames whose differences are within a predetermined range, calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames whose sharpness is lower than the preset value; and adjust the pixels, brightness and feature regions of other important image frames according to the reference image frames.

[0080] Specifically, the target region is extracted from all important image frames, including: obtaining the foreground and background regions and the unknown region Ii = αFi + (1-α)Bi from the important image frames, where α represents transparency, Fi is the foreground pixel, Bi is the background pixel, and i represents the i-th pixel. To make the points in the unknown region as close as possible to the foreground and background regions, the distance between the pixel color within the radius of each point in the unknown region and the pixel color of that point is obtained. Pixels with a distance greater than a set maximum threshold are designated as foreground regions, while pixels with a distance less than a set minimum threshold are designated as background regions, thus narrowing down the range of unknown areas. The values ​​obtained from back projection are compared with those obtained from orthographic projection. If the results are outside a predetermined range, it indicates that the image is very blurry, and the image is discarded.

[0081] Using formula Calculate the sharpness of each important image frame, where x is the number of pixels in length of the important image frame, y is the number of pixels in width of the important image frame, and z is the image size. The image with the highest sharpness is used as the reference image frame.

[0082] If any important image frames have a resolution lower than a preset value, then this data is adjusted for resolution. Specifically, this involves: identifying blurred regions within these important image frames with resolution below the preset value, reconstructing these blurred regions, and encoding the reconstructed image frames. Reconstructing the blurred regions specifically involves pixel reconstruction of these regions, reconstructing the pixel coordinates. n represents a point on a circle with the center of the starting region as the origin and the diagonal length of the starting region as the radius. Typically, n = 4 points are chosen as four points evenly distributed on the circle. d i R is the distance between the reconstructed coordinates of the selected starting region block and the circumferential pixel coordinates. jR1 represents the pixel value of the pixel coordinates of the selected point on the circumference. Specifically, d1 is the distance between the reconstructed coordinates of the selected starting region block and the pixel coordinates of the adjacent region block to the left, and R1 represents the pixel value of the pixel coordinates of the adjacent region block to the right; d2 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block to the right, and R2 represents the pixel value of the pixel coordinates of the adjacent region block to the left; d3 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block above, and R3 represents the pixel value of the pixel coordinates of the adjacent region block below; d4 is the distance between the reconstructed coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block below, and R4 represents the pixel value of the pixel coordinates of the adjacent region block above.

[0083] Pixel adjustment specifically includes: determining the pixel values ​​of the target region in other important image frames based on the back-projected images of those frames. Where n represents the number of projected lines in the image, q k,i This represents the i-th projection line passing through pixel k. The pixel ratio Z = Pv / h(k) between these target regions and the target region of the reference image frame is calculated, where PV is the pixel value of the reference image frame. Pixel scaling is then applied to other important image frames based on this pixel ratio. Brightness adjustment specifically includes: calculating the brightness of the reference image frame = R*0.299 + G*0.587 + B*0.114. The weights in the brightness calculation are adjusted according to actual needs. Brightness adjustment of other key image frames is then performed according to the brightness of the reference image frame. Feature region adjustment specifically includes: performing rotation operations and proportional scaling on other key image frames according to the reference image frame.

[0084] The deep learning module 24 is used to construct a target feature pattern model based on deep learning, and input the obtained high-resolution image into the target feature pattern model to determine the target feature pattern.

[0085] The deep learning module 24 is specifically used for: extracting target features from the obtained high-resolution image to form a target feature vector; inputting the target feature vector set into the target feature pattern model, training multiple damping trend prediction models to obtain different sub-trend prediction models, using each sub-trend prediction model to predict the trend of the target feature vector set, estimating the set of weights of each sub-prediction model through the trend prediction results; finding the optimal value corresponding to each weight in the set of weights, and determining the target feature pattern through the combination of the optimal values ​​of each sub-prediction model and its corresponding weight.

[0086] Specifically, the target pedestrian feature vector set is input into the target pedestrian feature trend model, and multiple damping trend prediction models are trained using the target pedestrian feature vector set, with each damping trend prediction model serving as a sub-prediction model. The sub-prediction model is then used to predict the feature vector set of the target pedestrian, and the prediction result is obtained. The prediction result is then used with a formula. The set of weights for estimating the sub-prediction model {λ1, λ2, λ3, ..., λ4} T}, where x i y represents the target shape feature in the target feature vector. i Let μ1 and μ2 be the target spatial relationship features of the target feature vector, μ1 and μ2 be the influence weights of the target shape features and the target spatial relationship features, n be the total number of target feature vectors, and T be the number of sub-prediction models.

[0087] Calculate each sub-prediction model The set of weights {λ1, λ2, λ3, ..., λ} T In the context of prediction models, the optimal value corresponds to each weight; through each sub-prediction model... And the optimal values ​​of their corresponding weights {λ1, λ2, λ3, ..., λ} T Combining to determine the target characteristics and patterns

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A deep learning-based image processing method, characterized in that, include: Receive video files transmitted from external terminals and perform security checks on the external terminals; Separate image frames from the video file and extract important image frames containing target features from the image frames; The system corrects important image frames, adjusts the sharpness of corrected important image frames, and outputs high-resolution images. A target feature pattern model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature pattern model to determine the target feature pattern. Performing a security check on external terminals includes the following sub-steps: Step 111: Extract external terminal information from the video file of the external terminal, including external terminal device information, data packet sending and receiving time, IP address, and port; Step 112: Determine whether the external terminal's IP address and port are allowed to be directly accessed by the server. If so, allow image processing of the video files of the external device; otherwise, proceed to step 113. Step 113: Calculate the security level of the external device and the server's access level to the external device. If the security level of the external device is greater than the server's access level, then image processing of the video file on the external device is allowed; otherwise, the external terminal request is rejected. Correcting important image frames involves the following sub-steps: Extract the target region from all important image frames, back-project the target region, compare the original image and the back-projected image to see if there are any differences, and select several important image frames whose differences are within a predetermined range. Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames with sharpness lower than the preset value; Adjust the pixels, brightness, and feature regions of other important image frames according to the reference image frame; A target feature pattern model is constructed based on deep learning. The obtained high-resolution image is then input into the target feature pattern model to determine the target feature pattern. The specific steps include the following: Step 141: Extract target features from the obtained high-resolution image to form a target feature vector; Step 142: Input the target feature vector set into the target feature pattern model, train multiple damping trend prediction models to obtain different sub-trend prediction models, use each sub-trend prediction model to predict the trend of the target feature vector set, and estimate the set of weights of each sub-prediction model through the trend prediction results. Specifically, the target pedestrian feature vector set is input into the target pedestrian feature trend model, and multiple damping trend prediction models are trained using the target pedestrian feature vector set, with each damping trend prediction model serving as a sub-prediction model. The sub-prediction model is then used to predict the feature vector set of the target pedestrian, and the prediction result is obtained. The prediction result is then used with a formula. The set of weights for estimating the sub-prediction model {λ1, λ2, λ3, ..., λ4} is given. T }, where x i y represents the target shape feature in the target feature vector. i denoted as the target spatial relationship feature of the target feature vector, μ1 and μ2 are the influence weights of the target shape feature and the target spatial relationship feature, n is the total number of target feature vectors, and T is the number of sub-prediction models; Step 143: Find the optimal value corresponding to each weight in the set of weights, and determine the target feature pattern by combining the optimal values ​​of each sub-prediction model and its corresponding weight. Specifically, calculate each sub-prediction model The set of weights {λ1, λ2, λ3, ..., λ} T In the context of prediction models, the optimal value corresponds to each weight; through each sub-prediction model... And the optimal values ​​of their corresponding weights {λ1, λ2, λ3, ..., λ} T Combining to determine the target characteristics and patterns 2. The image processing method based on deep learning as described in claim 1, characterized in that, Extract external terminal information from the video file, calculate the security level of the external terminal, and if the security level of the external device is higher than that of the server, allow image processing of the video file of the external device; otherwise, reject the external terminal request.

3. The image processing method based on deep learning as described in claim 1, characterized in that, Extracting key image frames containing target features from image frames includes the following sub-steps: From all the image frames separated from the video, find the first important image frame containing the target features; The target features in the first important image frame are processed into grayscale to determine the grayscale value of the target features; The frame distance between the image frame to be extracted and the first important image frame is calculated based on the grayscale distribution value of the target feature. All important image frames containing the target feature are then determined based on the frame distance.

4. The image processing method based on deep learning as described in claim 1, characterized in that, Extracting the target region from important image frames specifically includes: obtaining the foreground and background regions and unknown regions in the important image frames; in order to make the points in the unknown region as close as possible to the foreground and background regions, taking each point in the unknown region as the center, obtaining the distance between the pixel color in its radius and the pixel color of that point; taking the pixel distance greater than a set maximum threshold as the foreground region and taking the pixel distance less than a set minimum threshold as the background region, so as to narrow down the range of the unknown region.

5. A deep learning-based image processing system, characterized in that, include: The security check module is used to receive video files transmitted from external terminals and perform security checks on the external terminals. The important image frame determination module is used to separate image frames from video files and extract important image frames containing target features from the image frames; The high-resolution image output module is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images. The deep learning module is used to build a target feature pattern model based on deep learning, and input the obtained high-resolution image into the target feature pattern model to determine the target feature pattern; The security check module specifically includes extracting external terminal information from the video files of external terminals, including external terminal device information, data packet sending and receiving time, IP address, and port; determining whether the external terminal's IP address and port are allowed to be directly accessed by the server; if so, image processing of the external device's video files is allowed; otherwise, the security level of the external device is calculated, and the server's access openness to the external device is calculated; if the external device's security level is greater than the server's access openness, image processing of the external device's video files is allowed; otherwise, the external terminal's request is rejected. The high-resolution image output module specifically includes extracting the target region from all important image frames, back-projecting the target region, comparing the original image and the back-projected image to see if there are differences, and selecting several important image frames whose differences are within a predetermined range. Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on important image frames with sharpness lower than the preset value; Adjust the pixels, brightness, and feature regions of other important image frames according to the reference image frame; The deep learning module specifically includes extracting target features from the obtained high-resolution image to form a target feature vector; inputting the target feature vector set into the target feature pattern model, training multiple damping trend prediction models to obtain different sub-trend prediction models, using each sub-trend prediction model to predict the trend of the target feature vector set, and estimating the set of weights of each sub-prediction model through the trend prediction results. Specifically, the target pedestrian feature vector set is input into the target pedestrian feature trend model, and multiple damping trend prediction models are trained using the target pedestrian feature vector set, with each damping trend prediction model serving as a sub-prediction model. The sub-prediction model is then used to predict the feature vector set of the target pedestrian, and the prediction result is obtained. The prediction result is then used with a formula. The set of weights for estimating the sub-prediction model {λ1, λ2, λ3, ..., λ4} is given. T }, where x i y represents the target shape feature in the target feature vector. i Let μ1 and μ2 be the target spatial relationship features of the target feature vector, μ1 and μ2 be the influence weights of the target shape features and target spatial relationship features, n be the total number of target feature vectors, and T be the number of sub-prediction models. The optimal value for each weight in the set of weights is found, and the target feature pattern is determined by the combination of the optimal values ​​of each sub-prediction model and its corresponding weight. Specifically, the calculation of each sub-prediction model... The set of weights {λ1, λ2, λ3, ..., λ} T In the context of prediction models, the optimal value corresponds to each weight; through each sub-prediction model... And the optimal values ​​of their corresponding weights {λ1, λ2, λ3, ..., λ} T Combining to determine the target characteristics and patterns 6. The image processing system based on deep learning as described in claim 5, characterized in that, The security check module is specifically used to extract external terminal information from video files, calculate the security level of the external terminal, and allow image processing of the video files of the external device if the security level of the external device is higher than that of the server; otherwise, the external terminal request is rejected.

7. The image processing system based on deep learning as described in claim 5, characterized in that, The important image frame determination module is specifically used to find the first important image frame containing the target feature from all image frames separated from the video; to perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature; to calculate the frame distance between the image frame to be extracted and the first important image frame based on the grayscale distribution value of the target feature; and to determine all important image frames containing the target feature based on the frame distance.

8. The image processing system based on deep learning as described in claim 5, characterized in that, In the high-resolution image output module, the target region is extracted from the important image frame. Specifically, this is used to: obtain the foreground and background regions and unknown regions in the important image frame; in order to make the points in the unknown region as close as possible to the foreground and background regions, take each point in the unknown region as the center, obtain the distance between the pixel color in its radius and the pixel color of that point, and take the pixel distance greater than the set maximum threshold as the foreground region and the pixel distance less than the set minimum threshold as the background region, so as to narrow down the range of the unknown region.

Images