Factory video optimization acquisition method based on artificial intelligence

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an artificial intelligence-based factory video optimized acquisition method, which comprises the following steps of: installing monitoring cameras at different monitoring positions of a factory, and setting loss tolerance delta of sparse processing according to the monitoring positions and monitoring levels of acquisition periods of monitoring videos; all pixel points in the video frame subjected to the previous sparse processing are equally divided into two classes, four neighborhood pixel points of any pixel point are different from the class to which the pixel point belongs, and the class with the minimum loss degree forms the video frame subjected to the current sparse processing; and when the loss degree of the video frame after the Rth sparse processing is less than delta and the loss degree of the video frame after the (R + 1) th sparse processing is not less than delta, taking the video frame after the Rth sparse processing as an optimized acquisition result and transmitting the optimized acquisition result. According to the invention, the balance between the video quality and the compression degree is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to an optimized acquisition method for factory videos based on artificial intelligence. Background Art

[0002] A smart factory refers to an intelligent factory that establishes a virtual model and a simulation environment based on digital twin technology to optimize factory production and operation; through the digital twin smart factory, manufacturing enterprises can simulate the entire production process in a virtual environment, including equipment operation, material flow, personnel management, etc. This enables enterprises to better predict potential problems, optimize production processes, improve efficiency and quality, and at the same time reduce production costs.

[0003] In the construction of a smart factory, the acquisition of monitoring videos plays a crucial role in remote security monitoring. Through the acquisition and analysis of monitoring videos, a smart factory can achieve visual and intelligent management of the production process, improve production efficiency, safety and quality, reduce operating costs, and enhance the overall competitiveness of the enterprise; at the same time, the acquisition and transmission of monitoring videos are key links in realizing visual and intelligent management of the production process.

[0004] However, in practical applications, due to the large amount of monitoring video data, simultaneous acquisition at multiple locations will cause high network bandwidth pressure, resulting in transmission delays, affecting real-time performance, and further leading to missed opportunities for handling key events; therefore, it is necessary to compress and transmit the monitoring videos.

[0005] When compressing monitoring videos, excessive compression will reduce the quality of the monitoring videos. Insufficient monitoring video quality will prevent monitoring personnel from clearly identifying abnormal situations, affecting the reliability of monitoring; while insufficient compression will result in too large a data volume, increasing the transmission burden, affecting real-time performance, and further leading to missed opportunities for handling key events. Summary of the Invention

[0006] To solve the above technical problem of how to balance video quality and compression degree, the present invention provides an optimized acquisition method for factory videos based on artificial intelligence, including: installing monitoring cameras at different monitoring positions in the factory, and setting the loss tolerance of sparse processing according to the monitoring level of the monitoring position and the monitoring level of the acquisition period of the monitoring video ; performing multiple sparse processes on the video frames in the monitoring videos collected by the monitoring cameras until and ; taking the video frame as the optimized acquisition result and transmitting it, is the loss degree of the video frame after the th sparse process; the th sparse process includes: taking the​ The video frames after sub-sparse processing All pixel points in it are divided into two categories, and it is required that all neighboring pixel points within the 4-neighborhood of any pixel point are different from the category to which the pixel point belongs; calculate the loss degree when retaining any one category; form a video frame with all pixel points in the category with the smallest loss degree and use the smallest loss degree as the loss degree of the video frame Calculating the loss degree includes: calculating the contribution coefficient of each neighborhood position within the 4-neighborhood during the th sub-sparse processing according to the gray-scale difference between all pixel points in the non-target category of the two categories and their respective neighboring pixel points within the 4-neighborhood; according to the contribution coefficients of each neighborhood position within the 4-neighborhood during the previous sub-sparse processing and the gray-scale values of all pixel points in the target category, obtain a restored frame of the same size as the video frame through multiple restorations, and calculate the loss degree when retaining the target category according to the difference in the gray-scale values of the pixel points in the restored frame and the video frame .

[0007] The present invention performs multiple sub-sparse processing on the collected video frames, obtains an optimized acquisition result of the video frames and transmits them, realizes lossy compression of the video frames, effectively solves the problem of insufficient real-time performance caused by large data volume and high bandwidth pressure during the acquisition and transmission of monitoring videos in smart factories, ensures that the monitoring videos can be transmitted to the monitoring center in real time, and monitoring personnel can timely discover and handle abnormal situations in the production process, improving production efficiency and safety; in this process, according to the monitoring level of the monitoring location and the monitoring level of the acquisition period of the monitoring video, dynamically adjust the loss tolerance of the sub-sparse processing until a video frame with the maximum loss degree and less than the loss tolerance is obtained for the sub-sparse processed video frames, ensuring that the video quality under different monitoring requirements is within an acceptable range, reducing both the amount of data to be transmitted and retaining the key information in the monitoring videos, enabling monitoring personnel to clearly identify abnormal situations and ensuring the reliability of monitoring

[0008] Preferably, the loss tolerance of the sub-sparse processing , is the monitoring degree corresponding to the monitoring level of the monitoring location, is the monitoring degree corresponding to the monitoring level of the acquisition period of the monitoring video, is the upper limit of the loss tolerance; the monitoring levels include three levels, namely high level, medium level and low level, and the monitoring degrees corresponding to the three monitoring levels are respectively , , , and .

[0009] The present invention dynamically adjusts the loss tolerance of sparse processing according to the monitoring level of the monitoring position and the monitoring level of the acquisition period, and can ensure that the video quality meets the requirements under different monitoring needs.

[0010] Preferably, all pixel points in the video frame after the th sparse processing are divided into two categories, and it is required that all neighboring pixel points within the 4-neighborhood of any pixel point are different from the category to which the pixel point belongs, including: for all pixel points in the video frame , the pixel points where the row and column are odd rows and odd columns and the pixel points where the row and column are even rows and even columns are divided into the first category; the pixel points where the row and column are odd rows and even columns and the pixel points where the row and column are even rows and odd columns are divided into the second category.

[0011] Preferably, calculating the contribution coefficient of each neighborhood position within the 4-neighborhood during the th sparse processing includes: the contribution coefficient of the neighborhood position within the 4-neighborhood during the th sparse processing ; where is the average gray-scale difference of the neighboring pixel points at the neighborhood position within the 4-neighborhood during the th sparse processing, , is equal to , , , and the sum of

[0012] The present invention calculates the gray-scale difference between a pixel point and each neighboring pixel point within its 4-neighborhood, and calculates the contribution coefficient of each neighborhood position within the 4-neighborhood during sparse processing, so as to ensure that when the video frame is restored according to the contribution coefficient of each neighborhood position within the 4-neighborhood during sparse processing and the gray-scale values of the remaining pixel points, the difference between the restored frame and the original video frame is small, ensuring the video quality.

[0013] Preferably, obtaining a restored frame with the same size as the video frame through multiple restorations includes: setting a blank image with the same size as the video frame , from to 1; according to the positions of the retained pixel points in the video frame , setting all pixel points in the restored frame in the image to obtain the image to be restored ; through the The contribution coefficients of each neighborhood position in the 4-neighborhood during the sub-sparse processing are used to perform weighted summation on the gray values of all neighborhood pixels of each blank pixel in the image to be restored, so as to obtain the gray value of each blank pixel in the image to be restored, and then obtain the th restored frame; stop until is equal to 1, and the restored frame is obtained. The restored frame has the same size as the video frame . Preferably, obtaining the gray value of each blank pixel in the image to be restored includes: ; where

[0014] is the gray value of the blank pixel, is the number of neighborhood positions where neighborhood pixels exist in the 4-neighborhood of the blank pixel, is the th neighborhood position where neighborhood pixels exist in the 4-neighborhood of the blank pixel, is the contribution coefficient of the th neighborhood position, is the gray value of the th neighborhood pixel existing in the 4-neighborhood of the blank pixel, is the th neighborhood position where no neighborhood pixel exists in the 4-neighborhood of the blank pixel, is the contribution coefficient of the th neighborhood position, is the average value of the gray values of all neighborhood pixels existing in the 4-neighborhood of the blank pixel, represents rounding.

[0015] Preferably, the positions of the pixels retained in the video frame are determined according to the retention class in the video frame: when the retention class is the first class, the positions of the pixels to be retained refer to the positions where the row and column are odd rows and odd columns and the positions where the row and column are even rows and even columns; when the retention class is the second class, the positions of the pixels to be retained refer to the positions where the row and column are odd rows and even columns and the positions where the row and column are even rows and odd columns.

[0016] Preferably, calculating the loss degree when retaining the target class according to the difference in the gray values of the pixels in the restored frame and the video frame includes: taking the average value of the differences in the gray values of all pixels in the restored frame and all pixels in the video frame as the loss degree when retaining the target class.

[0017] ​Preferably, the method further includes: the number of times of sparse processing during the process of obtaining the optimized acquisition result , the retained classes in the video frames after each sparse processing, and the contribution coefficients of each neighborhood position in the 4-neighborhood during each sparse processing are used as supplementary information and transmitted.

[0018] The present invention transmits the supplementary information so that the monitoring center can accurately restore according to the supplementary information and the optimized acquisition result, ensuring the reliability of monitoring.

[0019] Preferably, the method further includes: in the monitoring center, according to the optimized acquisition result and the supplementary information, the video frames for display are obtained through multiple restorations, including: setting a blank image with a size equal to , as the intermediate image , and the intermediate image refers to the optimized acquisition result; according to the retained classes in the video frames after the th sparse processing, the positions of the retained pixel points in the video frames after the th sparse processing are determined, which are used to set the pixel points in the intermediate image in the blank image ; through the contribution coefficients of each neighborhood position in the 4-neighborhood during the th sparse processing, the gray values of all neighborhood pixel points of the remaining blank pixel points in are weighted and summed to obtain the gray values of the remaining blank pixel points, obtaining the nd intermediate image ; Taking from to 1 until the intermediate image is obtained as the video frame for display.

[0020] The beneficial effects of the present invention are as follows: By performing multiple sparse processings on the acquired video frames, the present invention obtains the optimized acquisition result of the video frames and transmits them, realizing the lossy compression of the video frames; at the same time, according to the monitoring level of the monitoring location and the monitoring level of the acquisition period of the monitoring video, the loss tolerance of the sparse processing is dynamically adjusted until the loss degree of the video frames obtained by the sparse processing is the largest and less than the loss tolerance, ensuring that the video quality under different monitoring requirements is within an acceptable range; therefore, the present invention not only improves the efficiency of monitoring video transmission, but also ensures the real-time performance and reliability of monitoring, providing strong support for the efficient operation and safety management of smart factories. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 ​is a flowchart schematically showing the method for optimizing the acquisition of factory videos based on artificial intelligence in the present invention; Figure 2 is a schematic diagram schematically showing the classification of pixel points; Figure 3 is a schematic diagram schematically showing a blank image; Figure 4 is a schematic diagram schematically showing an image to be restored; Figure 5 is a schematic diagram schematically showing a restored frame. Detailed implementation manners

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0023] Next, the detailed implementation manners of the present invention will be described in detail in conjunction with the accompanying drawings.

[0024] The embodiments of the present invention disclose a method for optimizing the acquisition of factory videos based on artificial intelligence. Referring to Figure 1 , it includes steps S1 to S3: S1. Install monitoring cameras at different monitoring positions in the factory to collect monitoring videos of each monitoring position; set the loss tolerance of sparse processing according to the monitoring level of the monitoring position and the monitoring level of the acquisition period of the monitoring video.

[0025] A smart factory refers to an intelligent factory that builds a virtual model and a simulation environment based on digital twin technology to optimize factory production and operation; through the digital twin smart factory, manufacturing enterprises can simulate the entire production process in a virtual environment, including equipment operation, material flow, personnel management, etc. This enables enterprises to better predict potential problems, optimize production processes, improve efficiency and quality, and at the same time reduce production costs.

[0026] In the construction of a smart factory, the acquisition of monitoring videos plays a crucial role in remote security monitoring; by transmitting the monitoring videos to a remote terminal through the network, managers can view the real-time pictures of the factory at any time through devices such as mobile phones and computers, realizing remote monitoring and management, and supporting collaborative work among multiple departments, improving work efficiency and collaborative effects; at the same time, the monitoring videos can monitor the personnel activities and environmental conditions in the factory in real time, ensure that employees comply with safety operation procedures, promptly discover and handle unsafe behaviors, and ensure the safety of personnel and equipment.

[0027] In a factory, setting multiple monitoring locations is crucial for achieving comprehensive monitoring and optimizing the production process. Common monitoring locations include: production equipment areas, material storage areas, personnel work areas, safety-critical areas, energy management areas, and environmental monitoring areas, and the monitoring levels are respectively: high, medium, low, high, medium, medium.

[0028] The acquisition period of the monitoring video can be divided according to the production plan and management requirements of the factory: usually divided into production periods and non-production periods. The production period includes from 8 am to 5 pm on weekdays, and the monitoring level is high; the non-production period includes from 5 pm on weekdays to 8 am the next day, and the whole day on weekends, and the monitoring level is medium.

[0029] The monitoring levels include three levels, namely high, medium, and low. The monitoring degrees corresponding to the three monitoring levels are respectively 、 、 ,and ; 、 、 The specific values of can be set according to the actual application scenarios and requirements, and the value range is [0, 1]. In this embodiment, 、 、 are respectively set to 0.95, 0.75, and 0.5.

[0030] Specifically, install monitoring cameras at different monitoring locations in the factory to collect monitoring videos of each monitoring location.

[0031] For the monitoring video collected by the monitoring camera, take any video frame in the monitoring video as the initial video frame, and denote it as video frame ; for video frame , taking the pixel point in the lower left corner of the video frame as the origin, the horizontal right direction from the origin as the positive direction of the horizontal axis, and the vertical upward direction from the origin as the positive direction of the vertical axis, construct a rectangular coordinate system, and denote the coordinates of the pixel points in the video frame in the rectangular coordinate system as , is the abscissa of the pixel point, is the ordinate of the pixel point.

[0032] Furthermore, according to the monitoring level of the monitoring location and the monitoring level of the acquisition period of the monitoring video, set the loss tolerance of the sparse processing; then the loss tolerance of the sparse processing , is the monitoring degree corresponding to the monitoring level of the monitoring location, is the monitoring degree corresponding to the monitoring level of the acquisition period of the monitoring video, is the upper limit of the loss tolerance.

[0033] Among them, the upper limit of the loss tolerance The specific value can be set according to the actual application scenario and requirements, and the value range is [40, 60]. In this embodiment, the upper limit of the loss tolerance Is set to 50.

[0034] It should be noted that according to the monitoring level of the monitoring location and the monitoring level of the acquisition period of the monitoring video, the loss tolerance of the sparse processing is dynamically adjusted to ensure the video quality under different monitoring requirements: for areas with a high monitoring level, a lower loss tolerance can be set to ensure the video quality; for areas with a low monitoring level, a higher loss tolerance can be set to further reduce the data volume.

[0035] S2. Perform multiple sparse processes on the initial video frames in the monitoring video until the loss degree of the video frames after the current sparse process is less than the loss tolerance and the loss degree of the video frames after the next sparse process is not less than the loss tolerance, and then use the video frames after the current sparse process as the optimized acquisition result and transmit it.

[0036] It should be noted that by performing multiple sparse processes on the video frames The number of pixel points in the video frames after each sparse process will be reduced by half, thereby reducing the size of the video frames to be transmitted. Therefore, in this embodiment, through sparse processing, lossy compression of the video frames is achieved, which can significantly reduce the data volume and reduce the demand for network bandwidth, enabling the video to be transmitted more smoothly, reducing latency and jitter, ensuring that the monitoring video can be transmitted to the monitoring center in real time, and further ensuring that the monitoring personnel can promptly discover and handle abnormal situations in the production process, improving production efficiency and safety.

[0037] Specifically, for the video frames , perform multiple sparse processes on the video frames until the loss degree of the video frames after the th sparse process and the loss degree of the video frames after the th sparse process, and use the video frames after the th sparse process as the optimized acquisition result. , are respectively the loss degrees of the video frames after the th sparse process and the video frames after the th sparse process.

[0038] It should be noted that in this embodiment, for the optimized acquisition result of the monitored video, the video frames with the largest loss degree and less than the loss tolerance are selected, so as to ensure that the video quality under different monitoring requirements is within an acceptable range. This not only reduces the amount of data to be transmitted but also retains the key information in the monitored video, enabling the monitoring personnel to clearly identify abnormal situations and ensuring the reliability of monitoring.

[0039] Furthermore, the number of times of sparse processing during the process of obtaining the optimized acquisition result , the retained classes in the video frames after each sparse processing, and the contribution coefficients of each neighborhood position in the 4-neighborhood during each sparse processing are used as supplementary information.

[0040] In addition, the optimized acquisition result and the supplementary information are transmitted to the monitoring center.

[0041] Among them, the acquisition step of the video frame after the th sparse processing is as follows: 1. Denote the video frame after the th sparse processing as . Divide all the pixel points in the video frame into two categories and denote them as the first category and the second category in the video frame respectively, requiring that all the neighborhood pixel points within the 4-neighborhood of any pixel point are different from the class to which the pixel point belongs.

[0042] Specifically, for all the pixel points in the video frame , divide the pixel points whose row number and column number are odd rows and odd columns and the pixel points whose row number and column number are even rows and even columns into the first category in the video frame ; divide the pixel points whose row number and column number are odd rows and even columns and the pixel points whose row number and column number are even rows and odd columns into the second category in the video frame .

[0043] Exemplarily, the schematic diagram of pixel point classification is as shown in Figure 2 , where the pixel points in the first category are marked as 1 and the pixel points in the second category are marked as 2.

[0044] Among them, the odd row / odd column refers to the row / column with an odd serial number; the even row / even column refers to the row / column with an even serial number.

[0045] For the pixel point with the coordinate , its 4-neighborhood consists of 4 neighborhood pixel points located above, below, left, and right of it. The 4 neighborhood pixel points above, below, left, and right of the pixel point refer to the coordinates , , , pixel points; at the same time, the four positions of up, down, left, and right in the 4-neighborhood are denoted as the four neighborhood positions in the 4-neighborhood.

[0046] 2. Calculate the loss degree when any class is retained in the video frame .

[0047] Specifically, after all pixel points in the video frame are divided into two categories, any one of the two categories in the video frame is used as the target class, then the other class in the video frame is used as the non-target class; according to the gray-scale differences between all pixel points in the non-target class in the video frame and their respective neighborhood pixel points in the 4-neighborhood, calculate the contribution coefficients of each neighborhood position in the 4-neighborhood during the th sparse processing; according to the contribution coefficients of each neighborhood position in the 4-neighborhood during the previous sparse processing and the gray-scale values of all pixel points in the target class, obtain a restored frame of the same size as the video frame through multiple restorations; according to the differences between the pixel points in the restored frame and the pixel points in the video frame , calculate the loss degree when the target class is retained.

[0048] The specific calculation process of the loss degree when the target class is retained is as follows: 2.1. According to the gray-scale differences between all pixel points in the non-target class in the video frame and their respective neighborhood pixel points in the 4-neighborhood, calculate the contribution coefficients of each neighborhood position in the 4-neighborhood during the th sparse processing.

[0049] For the 4-neighborhood, the four neighborhood pixel points in the 4-neighborhood are respectively denoted as neighborhood pixel point , neighborhood pixel point , neighborhood pixel point , neighborhood pixel point ; the four neighborhood positions in the 4-neighborhood are respectively denoted as neighborhood position , neighborhood position , neighborhood position , neighborhood position ; and the positions where neighborhood pixel point , neighborhood pixel point , neighborhood pixel point , neighborhood pixel point are located are respectively neighborhood position , neighborhood position , neighborhood position , neighborhood position .

[0050] Specifically, any pixel point among all pixel points in the non-target class in the video frame is used as the concerned pixel point; the gray-scale difference between the concerned pixel point and each neighboring pixel point within its 4-neighborhood is calculated, where the gray-scale difference refers to the absolute value of the difference in gray-scale values; thus, the gray-scale differences between each pixel point in the non-target class and each neighboring pixel point within its 4-neighborhood are calculated.

[0051] Further, according to the gray-scale differences between all pixel points in the non-target class in the video frame and different neighboring pixel points within their 4-neighborhoods, the contribution coefficients of different neighboring positions within the 4-neighborhood during the th sparse processing are calculated. Then, during the th sparse processing, the contribution coefficient of the neighboring position within the 4-neighborhood ; in the formula, is the average gray-scale difference of the neighboring pixel points at the neighboring position within the 4-neighborhood during the th sparse processing, , , is the average gray-scale difference of the neighboring pixel points at the neighboring position within the 4-neighborhood during the th sparse processing, ; represents the sum of , , , .

[0052] Among them, the average gray-scale difference of the neighboring pixel points at the neighboring position within the 4-neighborhood during the th sparse processing; in the formula, is the number of all pixel points in the non-target class in the video frame , is the gray-scale difference between the th pixel point in the non-target class and the neighboring pixel point .

[0053] It should be noted that the larger the average gray-scale difference of the neighboring pixel points at the neighboring position within the 4-neighborhood, the smaller the contribution coefficient of the neighboring position within the 4-neighborhood. In this way, when the video frame is restored based on the contribution coefficients of each neighboring position within the 4-neighborhood during the sparse processing and the gray-scale values of the retained pixel points, the difference between the restored frame and the original video frame is small, ensuring the video quality.

[0054] 2.2. According to the contribution coefficients of each neighborhood position in the 4-neighborhood during the previous sparse processing and the gray values of all pixel points in the target class, multiple restoration operations are performed to obtain a restored frame with the same size as the video frame.

[0055] The specific process of obtaining the restored frame through multiple restorations is as follows: (1) Set a blank image with the same size as the video frame . The schematic diagram is shown in Figure 3.

[0056] (2) According to the positions of the pixel points retained in the video frame , fill all the pixel points in the target class into the image to obtain the image to be restored . The schematic diagram is as Figure 4 shown.

[0057] Since the image to be restored has the same size as the video frame , and the number of all pixel points in the target class is equal to half of the number of all pixel points in the video frame , therefore, filling all the pixel points in the target class into the image results in the image to be restored still having half of its pixel points as blank pixel points, and the neighborhood pixel points within the 4-neighborhood of each blank pixel point are pixel points in the target class.

[0058] Additionally, since in this embodiment all the pixel points in the video frame are divided into two categories, and it is required that all the neighborhood pixel points within the 4-neighborhood of any pixel point are different from the class to which the pixel point belongs, therefore, for any pixel point in the non-target class in the video frame , the neighborhood pixel points within its 4-neighborhood are essentially pixel points in the target class in the video frame .

[0059] (3) Through the contribution coefficients of each neighborhood position in the 4-neighborhood during the previous sparse processing, perform weighted summation on the gray values of all neighborhood pixel points within the 4-neighborhood of each blank pixel point in the image to be restored to obtain the gray value of each blank pixel point in the image to be restored . In this way, the th restored frame is obtained. The schematic diagram is as Figure 5 shown.

[0060] (4) Set a blank image with the same size as the video frame , where From Take up to 1.

[0061] (5) According to the positions of the reserved pixel points in the video frame Set all the pixel points in the restored frame In the image To obtain the image to be restored .

[0062] Since the restored frame And the video frame Have the same size, the image to be restored Is equal in size to the video frame , while the size of the video frame Is half the size of the video frame , therefore, half of the pixel points in the image to be restored Are blank pixel points, and the neighboring pixel points within the 4-neighborhood of each blank pixel point are the pixel points in the restored frame .

[0063] (6) Through the contribution coefficients of each neighborhood position within the 4-neighborhood during the th sparse processing, weight-sum the gray values of all neighboring pixel points of each blank pixel point in the image to be restored To obtain the gray value of each blank pixel point in the image to be restored , thereby obtaining the th restored frame .

[0064] (7) And so on, until Equals 1, obtain the restored frame , the restored frame Is the same size as the video frame .

[0065] Among them, the positions of the reserved pixel points in the video frame are determined according to the reserved classes in the video frame: when the reserved class is the first class, the positions of the reserved pixel points refer to the positions where the row and column are odd rows and odd columns and where the row and column are even rows and even columns, and when the reserved class is the second class, the positions of the reserved pixel points refer to the positions where the row and column are odd rows and even columns and where the row and column are even rows and odd columns.

[0066] Among them, through the contribution coefficients of each neighborhood position within the 4-neighborhood during the th sparse processing, weight-sum the gray values of all neighboring pixel points of each blank pixel point in the image to be restored To obtain the gray value of each blank pixel point in the image to be restored , and the specific calculation formula is: ; In the formula, is the gray value of a blank pixel point, is the number of neighborhood positions where neighborhood pixel points exist within the 4-neighborhood of the blank pixel point, then represents the number of neighborhood positions where no neighborhood pixel points exist within the 4-neighborhood of the blank pixel point, is the th neighborhood position where a neighborhood pixel point exists within the 4-neighborhood of the blank pixel point 's contribution coefficient, is the gray value of the th neighborhood pixel point that exists within the 4-neighborhood of the blank pixel point, is the th neighborhood position where no neighborhood pixel point exists within the 4-neighborhood of the blank pixel point 's contribution coefficient, is the average value of the gray values of all neighborhood pixel points that exist within the 4-neighborhood of the blank pixel point, represents rounding.

[0067] Among them, when neighborhood pixel points exist at all neighborhood positions within the 4-neighborhood of the blank pixel point, the number of neighborhood positions where neighborhood pixel points exist within the 4-neighborhood of the blank pixel point = 4, then the number of neighborhood positions where no neighborhood pixel points exist within the 4-neighborhood of the blank pixel point

[0068] It should be specifically noted that for pixel points located in the first row / first column / last row / last column of the video frame, the number of neighborhood positions where neighborhood pixel points exist within their 4-neighborhood is less than 4. That is to say, their 4-neighborhood includes multiple neighborhood positions where no neighborhood pixel points exist. For the neighborhood positions where no neighborhood pixel points exist, in this embodiment, weighted summation is performed through the average value of the gray values of all neighborhood pixel points that exist within the 4-neighborhood of the blank pixel point to obtain the restored value of the blank pixel point.

[0069] 2.3. Calculate the loss degree when retaining the target class according to the difference between the gray values of the pixel points in the restored frame and the pixel points in the video frame .

[0070] Specifically, the average value of the differences between the gray values of all pixel points in the restored frame and all pixel points in the video frame is used as the loss degree when retaining the target class.

[0071] It should be noted that the greater the difference between the grayscale value and the restored value, the greater the loss degree when retaining the target class.

[0072] Therefore, after all pixel points in the video frame are evenly divided into two categories, calculate the loss degree when retaining each category.

[0073] 3. Take the category with the smallest loss degree as the retained category in the video frame ; Denote all pixel points in the retained category in the video frame as the retained pixel points in the video frame ; And form the video frame after the th sparse processing by all the retained pixel points in the video frame ; Take the smallest loss degree, that is, the loss degree of the retained category in the video frame , as the loss degree of the video frame after the th sparse processing.

[0074] It should be specifically noted that when , the steps to obtain the video frame after the th sparse processing are as follows: Evenly divide all pixel points in the initial video frame into two categories, requiring that all neighboring pixel points within the 4-neighborhood of any pixel point are different from the category to which the pixel point belongs; Take any category as the target category, then the other category is used as the non-target category; Calculate the loss degree when retaining the target category; Form the video frame after the th sparse processing by all pixel points in the category with the smallest loss degree, and take the smallest loss degree as the loss degree of the video frame after the th sparse processing.

[0075] S3. In the monitoring center, according to the optimized acquisition result and supplementary information, obtain the video frame for display through multiple restorations to achieve remote security monitoring.

[0076] The process of obtaining the video frame for display through multiple restorations is as follows: (1) Set a blank image with a size equal to ; Among them, ranges from to 1, is the number of times of sparse processing, is the size of the intermediate image ; When = , the intermediate image refers to the optimized acquisition result.

[0077] (2) Determine the th according to the retained category in the video frame after the The positions of the pixels retained in the video frame after the -th sparse processing; according to the positions of the pixels retained in the video frame after the -th sparse processing, set all the pixels in the -th intermediate image in the blank image ; through the contribution coefficients of each neighborhood position in the 4-neighborhood during the -th sparse processing, perform a weighted sum of the gray values of all the neighborhood pixels of the remaining blank pixels in the blank image to obtain the gray values of the remaining blank pixels, and thus obtain the -th intermediate image

[0078] (3)And so on, until equals 1, to obtain the intermediate image , which is used as the video frame for display.

[0079] It should be noted that according to the optimized acquisition results and supplementary information, the present invention obtains the video frame for display through multiple restorations, realizes the visualization and intelligent management of the production process. Managers can view the real-time pictures of the factory at any time through devices such as mobile phones and computers, realizing remote monitoring and management, which helps the collaborative work among multiple departments and improves work efficiency and collaborative effects.

Claims

1. The factory video optimization acquisition method based on artificial intelligence is characterized by: include: Install surveillance cameras at different monitoring locations in the factory, and set the loss tolerance for sparse processing according to the monitoring level of the monitoring location and the monitoring level of the surveillance video collection period. ; Video frames in the surveillance video collected by the surveillance camera Perform multiple sparse processing until and , the video frame As the optimization results are collected and transmitted, For the Video frame after sparse processing the extent of the loss; The second sparse processing includes: Video frame after sparse processing All pixels in the video are divided into two categories, requiring that all neighboring pixels in the four neighborhoods of any pixel are different from the class to which the pixel belongs; calculating the degree of loss when retaining any class; and composing all pixels in the class with the smallest degree of loss into a video frame. , and take the minimum loss as the video frame The loss degree is calculated by: calculating the loss degree according to the grayscale difference between all pixels in the non-target class in the two classes and each neighboring pixel in its 4 neighborhoods, The contribution coefficient of each neighborhood position in the 4 neighborhoods during the secondary sparse processing; The contribution coefficients of each neighborhood position in the 4 neighborhoods and the grayscale values ​​of all pixels in the target class during the secondary sparse processing are obtained by multiple restoration and the video frame The restored frames of the same size are based on the restored frames and video frames. The difference in the grayscale values ​​of the pixels in the image is used to calculate the degree of loss when retaining the target class.

2. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1 is characterized in that: The loss tolerance of the sparse processing , is the monitoring level corresponding to the monitoring level of the monitoring location, The monitoring level corresponding to the monitoring level during the acquisition period of the monitoring video, is the upper limit of loss tolerance; The monitoring levels include three levels, namely high, medium and low, and the monitoring degrees corresponding to the three monitoring levels are , , ,and .

3. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1 is characterized in that: The Video frame after sparse processing All pixels in the image are divided into two categories, requiring that all neighboring pixels in the 4-neighborhood of any pixel are different from the class to which the pixel belongs, including: For video frames For all the pixels in the image, the pixels whose rows and columns are odd-numbered rows and columns and the pixels whose rows and columns are even-numbered rows and columns are classified into the first category; the pixels whose rows and columns are odd-numbered rows and columns and the pixels whose rows and columns are even-numbered rows and columns are classified into the second category.

4. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1 is characterized in that: The calculation The contribution coefficient of each neighborhood position in the 4 neighborhoods during the secondary sparse processing includes: No. Neighborhood positions in 4 neighborhoods during secondary sparse processing The contribution coefficient ; In the formula, For the Neighborhood positions in 4 neighborhoods during secondary sparse processing The mean grayscale difference of the neighboring pixels at , , equal , , , The sum of.

5. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1 is characterized in that: The video frames are obtained by multiple restorations. Restored frames of the same size, including: Set a and video frame A blank image of equal size , from Get 1; Based on the video frame The position of the retained pixel in the frame will be restored All pixels in the image are set , obtain the image to be restored ; Through the The contribution coefficient of each neighborhood position in the 4 neighborhoods during the secondary sparse processing is used to restore the image. The grayscale values ​​of all neighboring pixels of each blank pixel in the image are weighted summed to obtain the image to be restored. The grayscale value of each blank pixel in the Restored Frames ; until Stop when it is equal to 1 and get the restored frame , restore frame With video frame Same size.

6. The method for optimizing factory video acquisition based on artificial intelligence according to claim 5 is characterized in that: The image to be restored is obtained The grayscale value of each blank pixel in includes: ； In the formula, is the gray value of the blank pixel, is the number of neighboring pixel locations in the 4-neighborhood of the blank pixel. There is a neighboring pixel in the 4-neighborhood of a blank pixel. Neighborhood locations The contribution coefficient of is the first pixel in the 4-neighborhood of the blank pixel. The gray value of the neighboring pixels, There is no neighboring pixel in the 4 neighborhoods of the blank pixel. Neighborhood locations The contribution coefficient of is the mean of the grayscale values ​​of all neighboring pixels in the 4-neighborhood of the blank pixel. Indicates rounding.

7. The method for optimizing factory video acquisition based on artificial intelligence according to claim 5 is characterized in that: The video frame The positions of the reserved pixels in the video frame are determined according to the reserved class in the video frame: when the reserved class is the first class, the positions of the reserved pixels refer to the positions where the rows and columns are odd rows and odd columns and the positions where the rows and columns are even rows and even columns; when the reserved class is the second class, the positions of the reserved pixels refer to the positions where the rows and columns are odd rows and even columns and the positions where the rows and columns are even rows and odd columns.

8. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1 is characterized in that: The restored frame and the video frame The difference in the grayscale values ​​of the pixels in the image is used to calculate the degree of loss when retaining the target class, including: Will restore the frame All pixels and video frames in The mean of the grayscale value differences of all pixels in is used as the degree of loss when retaining the target class.

9. The method for optimizing factory video acquisition based on artificial intelligence according to claim 1, characterized in that: The method further comprises: The number of times sparse processing is performed in the process of obtaining optimized collection results , the retained classes in the video frames after each sparse processing, and the contribution coefficients of each neighborhood position in the 4 neighborhoods during each sparse processing are transmitted as supplementary information.

10. The method for optimizing factory video acquisition based on artificial intelligence according to claim 9 is characterized in that: The method further includes: in the monitoring center, according to the optimized acquisition results and the supplementary information, obtaining a video frame for display by multiple restorations, including: setting a size equal to Blank image of , The intermediate image The size of the intermediate image Refers to optimizing the collection results; according to The retained classes in the video frames after the sparse processing are determined The positions of the pixels retained in the video frame after the secondary sparse processing are used to convert the intermediate image The pixels in the blank image are set in; through The contribution coefficient of each neighborhood position in the 4 neighborhoods during the secondary sparse processing is The grayscale values ​​of all neighboring pixels of the remaining blank pixels in the weighted summation are obtained to obtain the grayscale values ​​of the remaining blank pixels. Intermediate images ; from Get to 1 until you get the intermediate image , as the video frame for display.

Images