Industrial scene-oriented multi-task picture automatic compression backup method

Abstract

The invention provides an industrial scene-oriented multi-task picture automatic compression backup method, and aims to solve the problems of single-point failure, data coupling, low deployment efficiency and the like of a single-process multi-thread architecture of an existing industrial monitoring system. Physical isolation of each monitoring task is realized by generating a trinity isolation module containing an exclusive executable body, a configuration file and a batch processing script; an exclusive SQLite database and a resource pool are configured for each task, and independent and stable operation of the tasks is guaranteed in combination with triple conflict protection mechanisms of SHA-256 directory hash verification, process namespace detection and file lock mutual exclusion. The system can automatically complete self-starting deployment, industrial picture differential compression, storage according to date attribution and abnormity early warning, effectively improves fault isolation, IO performance and deployment efficiency, greatly saves storage space, is suitable for various industrial scenes with industrial pictures such as wind power equipment monitoring and electronic component detection, and meets the requirements of efficient storage and stable operation and maintenance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial image storage, in particular to a multi-task picture automatic compression backup method for industrial scenes. BACKGROUND

[0002] In the field of industrial production, industrial machines continuously output a large number of industrial pictures, which need to be stored and backed up for fault tracing, quality control and process optimization, so the industrial monitoring system becomes a key support.

[0003] The existing industrial monitoring system generally adopts a central controller architecture, and the core components include a central configuration database, a task scheduling engine and a shared storage system. Among them, the central configuration database stores the configuration parameters of all monitoring tasks, the task scheduling engine runs in a single process form, manages multiple monitoring tasks through internal threads, and the shared storage system provides all tasks with the same database to store image indexes. The working process is as follows: after the user creates multiple monitoring tasks through the configuration interface, the main process loads all task configurations into the memory, monitors the changes of each target folder through the polling thread mechanism, calls the processing module when a new file is detected, and finally stores the processing result in the shared index database.

[0004] However, the existing technology has the following defects: first, the risk of single point failure is high, once the main process crashes, all monitoring tasks will be interrupted at the same time, causing the interruption of industrial picture storage and affecting subsequent tracing work; second, the data coupling problem is prominent, all tasks share the shared database, which easily causes index conflicts and causes data confusion or loss; third, the deployment efficiency is low, the self-starting items of each device need to be configured manually, and when the number of devices or the size of tasks increases, the deployment workload increases significantly; fourth, resource preemption is serious, multiple tasks compete for database connections, causing uneven allocation of CPU, IO and other resources, affecting task processing efficiency; fifth, the operation and maintenance complexity is high, and the life cycle of a single task cannot be controlled independently, so when starting, stopping and maintaining a task, all tasks need to be suspended, affecting the operation of the whole monitoring system. These problems seriously restrict the stability, efficiency and operation convenience of industrial picture storage and backup, and it is difficult to meet the large-scale and high-reliability storage requirements in industrial scenes. SUMMARY

[0005] In view of the above technical deficiencies, the purpose of the present application is to provide a multi-task picture automatic compression backup method for industrial scenes, which improves fault isolation, IO performance and deployment efficiency.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] A multi-task picture automatic compression backup method for industrial scenes, comprising the following steps:

[0008] Step 1, receiving an industrial picture monitoring task created by a user, the industrial picture monitoring task corresponding to a target folder of an industrial machine;

[0009] Step 2, automatically generating a trinity module corresponding to the monitoring task, the trinity module including an exclusive executable body, an exclusive configuration file and an exclusive batch script;

[0010] Step 3, deploying the exclusive batch script to a system self-starting directory;

[0011] Step 4, starting the exclusive executable body through the exclusive batch script when the system starts;

[0012] Step 5, the exclusive executable body loading the exclusive configuration file, and creating an exclusive SQLite database of the monitoring task in a pre-device backup directory;

[0013] Step 6, initializing file system monitoring to monitor newly added industrial pictures in the target folder in real time;

[0014] Step 7, performing automatic compression processing on the newly added industrial pictures, and storing the compressed industrial pictures and corresponding index information in the exclusive SQLite database, to realize distributed independent running and data isolation of multiple tasks.

[0015] Preferably, in step 2, the generated exclusive batch script is further configured with a task start priority parameter, and when starting the exclusive executable body, the exclusive batch script sets the running priority of the exclusive executable body to be lower than the normal running priority of the system through a system process scheduling interface. The specific level of the running priority is determined according to the CPU resource occupation threshold of the industrial machine, and when the CPU utilization of the industrial machine exceeds the preset resource protection threshold, the exclusive batch script further reduces the running priority of the exclusive executable body, to avoid the monitoring task and the core production task of the industrial machine from occupying CPU resources, and to ensure the stable execution of the production process of the industrial machine.

[0016] Preferably, in step 7, before performing the automatic compression processing, a first conflict prevention step is further included: performing directory hash verification on the target folder of the newly added industrial pictures, calculating the current directory structure hash value of the target folder by using a SHA-256 hash algorithm, and comparing the hash value with a historical hash value obtained by last verification. If the two hash values are consistent, it is determined that there is no new effective file in the target folder, and the subsequent compression processing step is not performed; if the two hash values are inconsistent, the specific path of the new file is located through the hash difference value, and only the new file is subjected to the subsequent compression processing step, to avoid the problems of resource waste and data redundancy caused by repeated processing.

[0017] Preferably, the conflict protection further comprises a second safeguard step: immediately after the start of the exclusive executable, an exclusive.lock file corresponding to the monitoring task is created under the pre-device directory, and the exclusive.lock file is used to mark the running state of the monitoring task. When detecting that multiple monitoring tasks are associated with the same target folder, it is judged whether there is a task running conflict by detecting whether the exclusive.lock file corresponding to each monitoring task exists. If it is detected that there is an exclusive.lock file of a running task, the execution of the current monitoring task is suspended, and the state of the exclusive.lock file is detected again every preset time interval until the running task completes processing and releases the exclusive.lock file, and then the compression and backup process of the current monitoring task is started.

[0018] Preferably, the conflict protection further comprises a third safeguard step: a unique process namespace identifier is assigned to the exclusive executable of each monitoring task, and the process namespace identifier corresponds one-to-one to the name of the monitoring task. Before the file system monitoring in step 6 is initialized, the exclusive executable first detects whether there is a running process in the current system that is duplicated with the process namespace identifier of the exclusive executable. If a duplicated process is detected, the duplicated process is terminated through a system process management interface, and the file monitoring process of the exclusive executable is started; if no duplicated process is detected, the file monitoring process is directly started, and complete isolation of different monitoring tasks at the process level is achieved.

[0019] Preferably, in step 7, the automatic compression processing further comprises a differentiated compression step based on a preset MaxSize parameter: the MaxSize parameter is set in the exclusive configuration file, and the MaxSize parameter is a demarcation reference value for dividing the size of the industrial picture. When the size of the newly added industrial picture file monitored is greater than the MaxSize parameter, the first compression quality parameter is used to compress the industrial picture, and the first compression quality parameter is a preset low compression loss parameter suitable for large pictures, ensuring that the large picture after compression still meets the clarity requirement of industrial detection; when the size of the newly added industrial picture file is less than or equal to the MaxSize parameter, the second compression quality parameter is used to compress the industrial picture, and the second compression quality parameter is a preset high compression efficiency parameter suitable for small pictures, further improving the storage saving rate of small pictures, wherein the value of the first compression quality parameter is higher than the value of the second compression quality parameter.

[0020] Preferably, in step 7, a file date attribution division step based on a preset dayHour parameter is further included: the dayHour parameter is preset in the exclusive configuration file, which is an integer between 1 and 24, used to determine the storage date attribution of the industrial picture. When it is monitored that the generation time of the new industrial picture is between 0 o'clock of the current date and the time corresponding to the dayHour parameter, the storage date attribution of the industrial picture is determined as the previous day, and the index partition corresponding to the previous day is created in the exclusive SQLite database; when the generation time of the new industrial picture is between the time corresponding to the dayHour parameter and 24 o'clock of the current date, the storage date attribution of the industrial picture is determined as the current date, and the index partition corresponding to the current date is created in the exclusive SQLite database, realizing the ordered storage and fast retrieval of the industrial picture by date.

[0021] Preferably, an abnormality early warning step is further included: an abnormality monitoring module is integrated in the exclusive executable body, which monitors the running state of the exclusive executable body, the compression processing progress of the industrial picture, and the read-write state of the exclusive SQLite database in real time. When it is monitored that the exclusive executable body appears process unresponsive, compression processing timeout exceeds the preset time length, or database read-write fails, the abnormality monitoring module immediately generates abnormality information, which includes the abnormality type, the abnormality occurrence time, the corresponding monitoring task name, and the abnormality occurrence position. The abnormality monitoring module sends the generated abnormality information to the preset contact terminal, and at the same time creates a log file in the system local, records the complete abnormality details to the log file, and the log file is named according to the monitoring task name and date, which is convenient for subsequent query and troubleshooting by the operation and maintenance personnel.

[0022] Preferably, in step 7, when the automatic compression processing is performed on the new industrial picture, a memory cache optimization step is further included: when the exclusive executable body is started, according to the preset cache capacity parameter in the exclusive configuration file, an exclusive memory cache area of the monitoring task is initialized. When the new industrial picture is monitored, the file data of the new industrial picture is first loaded to the exclusive memory cache area, and then the compression processing operation is completed in the memory cache area. After the compression processing is completed, the compressed industrial picture data is written from the memory cache area to the pre-device directory, and the corresponding index information is written to the exclusive SQLite database. In the compression processing process, the exclusive executable body monitors the occupancy rate of the exclusive memory cache area in real time, and when the occupancy rate exceeds the preset cache threshold, the cache space occupied by the picture data that has completed the compression processing is automatically released, avoiding memory overflow affecting the normal operation of the monitoring task.

[0023] Preferably, a task life cycle independent control step is further included: a dedicated life cycle control instruction is configured for each monitoring task, and the life cycle control instruction includes a task start instruction, a task pause instruction, a task restart instruction and a task termination instruction. When an operation needs to be performed on a certain monitoring task, the corresponding life cycle control instruction is input, and the instruction only acts on the dedicated executable body of the monitoring task. When the task pause instruction is executed, the dedicated executable body stops the file monitoring and compression processing operation, and saves the current task progress to the dedicated configuration file; when the task restart instruction is executed, the dedicated executable body loads the task progress saved in the dedicated configuration file, and resumes the file monitoring and compression processing operation; when the task termination instruction is executed, the dedicated executable body first deletes the corresponding dedicated.lock file, and then releases the CPU resources and memory resources occupied by the task, thereby realizing independent operation and maintenance of a single monitoring task without affecting the operation of other monitoring tasks.

[0024] Compared with the prior art, the present application has the following advantages:

[0025] The present application adopts a multi-process distributed architecture, and each monitoring task is run in an independent process space through PID isolation. Each task is equipped with a dedicated executable body and a resource pool, thereby completely solving the problem of "single-process crash leading to interruption of all tasks" in the prior art. The fault isolation is improved by 100% compared with the traditional scheme. Even if an abnormality occurs in a certain task, other tasks can still operate normally, thereby ensuring uninterrupted industrial picture storage and backup.

[0026] The traditional centralized shared database is abandoned, and a dedicated SQLite database instance is configured for each task to realize data physical isolation and avoid index conflict. Meanwhile, in combination with the index partition mechanism according to the date, the data read / write speed is greatly improved. According to experimental verification, the IO performance is improved by 300% compared with the traditional scheme, the image retrieval time is significantly shortened, and the requirement for fast tracing of picture data in an industrial scene is met.

[0027] Automatic deployment is realized through a trinity isolation module. The system can automatically generate a dedicated executable body, a configuration file and a batch processing script, and deploy the batch processing script to a system self-starting directory, without the need for manual configuration of a self-starting item. The deployment efficiency is improved by 10 times compared with the traditional manual configuration method. Meanwhile, independent life cycle control of a single task is supported, and a certain task can be individually started, paused, restarted or terminated without affecting other tasks. The operation and maintenance complexity is greatly reduced, and the workload of operation and maintenance personnel is reduced.

[0028] The innovative triple conflict protection mechanism includes SHA-256 directory hash verification to accurately locate newly added files and avoid duplicate processing; process namespace detection to prevent duplicate processes from occupying resources; and file lock mutual exclusion mechanism to prevent multiple tasks from operating on the same resource at the same time. The triple protection reduces the task conflict rate to 0%, completely solving the processing anomaly problem caused by resource competition in traditional solutions.

[0029] By adopting a dedicated resource pool allocation mode, each task is allocated independent CPU and memory resources to avoid resource contention, and CPU utilization is improved by 65% ​​compared with the traditional competitive mode. Through a differentiated compression algorithm based on the MaxSize parameter, combined with data index optimization, storage space can be saved by more than 78%, which significantly reduces the storage hardware cost in industrial scenarios, while reducing resource consumption during data transmission and storage.

[0030] The integrated anomaly warning module can monitor the task running status, compression processing progress, and database read / write status in real time. Once a process becomes unresponsive, a processing timeout occurs, or a database anomaly occurs, detailed anomaly information is immediately generated and sent to a preset terminal. At the same time, local logs are recorded, which helps maintenance personnel to quickly locate and troubleshoot faults, shortens fault handling time, and further ensures the stable operation of the system. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0032] The invention will now be further described with reference to the accompanying drawings.

[0033] like Figure 1 As shown, a multi-task automatic image compression and backup method for industrial scenarios includes the following steps:

[0034] Step 1: Receive the industrial image monitoring task created by the user. The industrial image monitoring task corresponds to the target folder of the industrial machine.

[0035] Step 2: Automatically generate the three-in-one module corresponding to the monitoring task. The three-in-one module includes a dedicated executable, a dedicated configuration file, and a dedicated batch script.

[0036] Step 3: Deploy the dedicated batch script to the system's startup directory;

[0037] Step 4: When the system starts, a dedicated executable is launched via a dedicated batch script;

[0038] Step 5: The dedicated executable loads the dedicated configuration file and creates a dedicated SQLite database for the monitoring task in the preset backup directory;

[0039] Step 6, initialize the file system monitoring, real-time monitoring of the newly added industrial pictures in the target folder;

[0040] Step 7, performing automatic compression processing on the newly added industrial pictures monitored, and storing the compressed industrial pictures and corresponding index information into the exclusive SQLite database, realizing multi-task distributed independent running and data isolation.

[0041] Further, in step 2, the generated exclusive batch processing script is also configured with a task start priority parameter. When starting the exclusive executable, the exclusive batch processing script sets the running priority of the exclusive executable to be lower than the normal running priority of the system through the system process scheduling interface. The specific level of the running priority is determined according to the CPU resource occupation threshold of the industrial machine. When the CPU utilization of the industrial machine exceeds the preset resource protection threshold, the exclusive batch processing script will further reduce the running priority of the exclusive executable, avoiding the monitoring task and the core production task of the industrial machine from occupying CPU resources, and ensuring the stable execution of the production process of the industrial machine

[0042] Further, in step 7, before performing the automatic compression processing, a first conflict protection step is included: performing directory hash verification on the target folder where the newly added industrial pictures are monitored, calculating the current directory structure hash value of the target folder using the SHA-256 hash algorithm, and comparing the hash value with the historical hash value obtained by the last verification. If the two hash values are consistent, it is determined that there is no new effective file in the target folder, and the subsequent compression processing step is not performed; if the two hash values are inconsistent, the hash difference is used to locate the specific path of the new file, and only the new file is executed in the subsequent compression processing step, avoiding the problem of resource waste and data redundancy caused by repeated processing.

[0043] Further, the conflict protection also includes a second protection step: after the exclusive executable is started, an exclusive.lock file corresponding to the monitoring task is created in the pre-device directory. The exclusive.lock file is used to mark the running state of the monitoring task. When detecting that there are multiple monitoring tasks associated with the same target folder, it is judged whether there is a task running conflict by detecting whether the exclusive.lock file corresponding to each monitoring task exists. If an exclusive.lock file of a running task is detected, the execution of the current monitoring task is suspended, and the exclusive.lock file state is detected again every preset time interval until the running task completes processing and releases the exclusive.lock file, and then the compression backup process of the current monitoring task is started.

[0044] Further, the conflict protection further includes a third guarantee step: assigning a unique process namespace identifier to each monitoring task's exclusive executable, and the process namespace identifier corresponds to the name of the monitoring task. Before initializing the file system monitoring in step 6, the exclusive executable first detects whether there is a running process in the current system that duplicates the process namespace identifier of the exclusive executable. If a duplicate process is detected, the duplicate process is terminated through the system process management interface, and the file monitoring process of the exclusive executable is started; if no duplicate process is detected, the file monitoring process is directly started, and complete isolation of different monitoring tasks at the process level is achieved.

[0045] Further, in step 7, the automatic compression process further includes a differentiated compression step based on a preset MaxSize parameter: the MaxSize parameter is set in the exclusive configuration file, and the MaxSize parameter is a size demarcation value for dividing industrial pictures. When the size of the newly added industrial picture file monitored is greater than the MaxSize parameter, the first compression quality parameter is used to compress the industrial picture, and the first compression quality parameter is a preset low compression loss parameter suitable for large pictures, to ensure that the large picture after compression still meets the clarity requirement of industrial detection; when the size of the newly added industrial picture file is less than or equal to the MaxSize parameter, the second compression quality parameter is used to compress the industrial picture, and the second compression quality parameter is a preset high compression efficiency parameter suitable for small pictures, to further improve the storage saving rate of small pictures, wherein the value of the first compression quality parameter is higher than the value of the second compression quality parameter.

[0046] Further, in step 7, it further includes a file date attribution division step based on a preset dayHour parameter: the dayHour parameter is set in the exclusive configuration file, and the dayHour parameter is an integer between 1 and 24, used to determine the storage date attribution of the industrial picture. When the generation time of the newly added industrial picture is detected to be between 0 o'clock of the current date and the time corresponding to the dayHour parameter, the storage date attribution of the industrial picture is determined to be the previous day, and the index partition corresponding to the previous day is created in the exclusive SQLite database; when the generation time of the newly added industrial picture is between the time corresponding to the dayHour parameter and 24 o'clock of the current date, the storage date attribution of the industrial picture is determined to be the current date, and the index partition corresponding to the current date is created in the exclusive SQLite database, to achieve ordered storage and fast retrieval of industrial pictures by date.

[0047] Further, an abnormality early warning step is further included: an abnormality monitoring module is integrated in the exclusive executable body, the abnormality monitoring module monitors the running state of the exclusive executable body, the industrial picture compression processing progress and the read-write state of the exclusive SQLite database in real time. When it is monitored that the exclusive executable body appears process unresponsive, compression processing timeout exceeds the preset time length or database read-write fails, the abnormality monitoring module immediately generates abnormality information, the abnormality information includes the abnormality type, the abnormality occurrence time, the corresponding monitoring task name and the abnormality occurrence position. The abnormality monitoring module sends the generated abnormality information to the preset contact terminal, and simultaneously creates a log file in the system local, and records the complete abnormality details to the log file, the log file is named according to the monitoring task name and the date, so as to facilitate subsequent query and troubleshooting of the operation and maintenance personnel.

[0048] Further, in step 7, when the automatic compression processing is performed on the newly added industrial picture, a memory cache optimization step is further included: when the exclusive executable body is started, according to the preset cache capacity parameter in the exclusive configuration file, an exclusive memory cache area of the monitoring task is initialized. When the newly added industrial picture is monitored, the file data of the newly added industrial picture is first loaded to the exclusive memory cache area, and then the compression processing operation is completed in the memory cache area. After the compression processing is completed, the compressed industrial picture data is written from the memory cache area to the pre-device directory, and the corresponding index information is written to the exclusive SQLite database. In the compression processing process, the exclusive executable body monitors the occupancy rate of the exclusive memory cache area in real time, and when the occupancy rate exceeds the preset cache threshold, the cache space occupied by the picture data which has completed the compression processing is automatically released, so as to avoid memory overflow affecting the normal operation of the monitoring task.

[0049] Further, a task life cycle independent control step is further included: an exclusive life cycle control instruction is configured for each monitoring task, the life cycle control instruction includes a task start instruction, a task pause instruction, a task restart instruction and a task termination instruction. When it is needed to operate a certain monitoring task, the corresponding life cycle control instruction is input, and the instruction only acts on the exclusive executable body of the monitoring task. When the task pause instruction is executed, the exclusive executable body stops the file monitoring and compression processing operation, and saves the current task progress to the exclusive configuration file; when the task restart instruction is executed, the exclusive executable body loads the task progress saved in the exclusive configuration file, and recovers the file monitoring and compression processing operation; when the task termination instruction is executed, the exclusive executable body first deletes the corresponding exclusive.lock file, and then releases the CPU resources and memory resources occupied by the task, so as to realize independent operation and maintenance of a single monitoring task, and does not affect the operation of other monitoring tasks.

[0050] Embodiment one: application in the wind power equipment state monitoring scene

[0051] This embodiment is applied to the state monitoring system of 100 wind turbines in a certain wind farm. Each wind turbine is equipped with three industrial cameras to collect images of the wind turbine blade surface, the internal mechanical components of the nacelle, and the tower connection. Every 5 minutes, one BMP format industrial picture with a resolution of 4096x3072 is generated. All pictures need to be automatically compressed and backed up to meet the requirements of device fault tracing and storage cost control.

[0052] 1. System deployment

[0053] Based on the method of the present application, the user creates an independent "wind turbine XX state monitoring" task for each wind turbine through the front-end interface (a total of 100 tasks). In the task configuration, the target folder is set as the image output path of each wind turbine industrial camera (such as "D:\WindTurbine\Camera1\Data"), and the backup directory is set as the "E:\WindBackup\TaskXX" path of the local storage server of the wind farm.

[0054] The system automatically generates a trinity module for each task: "wind turbine XX state monitoring.exe" (exclusive executive body), "tasklist\wind turbine XX state monitoring.txt" (configuration file), and "tasklist\wind turbine XX state monitoring.bat" (batch processing script). The configuration file has the following preset parameters: MaxSize = 5MB (the original size of a BMP format 4096x3072 image is about 36MB, which is greater than 5MB), dayHour = 8 (8 o'clock is the date attribution dividing point), the first compression quality parameter = 85 (low loss compression for large images), the second compression quality parameter = 60 (high compression efficiency for small images), and the cache capacity parameter = 200MB; the batch processing script is configured with a startup priority of "lower than normal" to avoid competing for CPU resources with the core control program of the wind turbine.

[0055] The system automatically copies the batch processing scripts of the 100 tasks to the "C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\StartMenu\Programs\Startup" directory of the wind farm monitoring server, completing the self-starting deployment.

[0056] 2. Running process

[0057] When the server is restarted daily, the 100 batch processing scripts automatically run, respectively starting the corresponding exclusive executive body. After each executive body is started, it loads its own configuration file, creates an "index.db" exclusive SQLite database in the corresponding backup directory, and generates a "wind turbine XX state monitoring.lock" file in the backup directory to mark the running state, while detecting the process namespace to ensure no duplicate processes.

[0058] After initializing the file system monitoring, the execution unit monitors the target folder in real time. When a wind turbine industrial camera generates a blade surface image at 7:55, the execution unit detects a change in the target folder through SHA-256 hash verification, locates the newly added file path, and first loads the image data into a 200MB dedicated memory cache area. Because the image size of 36MB is greater than MaxSize=5MB, it is compressed using the first compression quality parameter of 85, resulting in a compressed image size of approximately 4.2MB. Since the generation time of 7:55 is earlier than dayHour=8, the storage date is determined to belong to the previous day. Information such as the image storage path, size before and after compression, generation time, and camera number are recorded in the previous day's index partition of the dedicated SQLite database. Subsequently, the compressed image is written to the backup directory, releasing the memory cache space.

[0059] If the monitoring tasks of two wind turbines trigger file processing at the same time, the system avoids conflicts through the .lock file mutual exclusion mechanism: the executor of the first triggered task holds the .lock file, the task that triggers later detects the existence of the .lock file, and re-detects it every 10 seconds until the first triggered task completes processing and deletes the .lock file, and then executes its own compression and backup process.

[0060] 3. Effect Verification

[0061] In a concurrent test with thousands of tasks (100 tasks), the system failure rate was 0%, and the average response latency was 23ms, meeting the real-time monitoring requirements of wind farms. In terms of storage optimization, the original 36MB BMP image was compressed to about 4.2MB. Combined with SQLite database index partitioning, the storage usage per task per day (288 images) was reduced from 10.368GB to 1.2096GB, saving about 88.3% of storage space, far exceeding the effect of "saving more than 78% of storage space" in the invention disclosure.

[0062] When a wind turbine camera disconnects, causing image generation to time out (exceeding 30 minutes), the anomaly monitoring module immediately generates anomaly information (anomaly type: image acquisition timeout, occurrence time: 2025-XX-XX 14:25, task name: wind turbine 32 status monitoring, anomaly location: D:\WindTurbine\Camera2\Data), sends it to the maintenance personnel's mobile phone via SMS, and records complete details in the log file "E:\WindBackup\Log\Wind Turbine 32 Status Monitoring_2025XX.txt". The maintenance personnel complete the fault diagnosis within 15 minutes, ensuring the stable operation of the monitoring system.

[0063] Example 2: Application in the scenario of BGA solder joint inspection for electronic components

[0064] This embodiment is applied to the BGA chip welding production line of an electronic factory. The production line is equipped with 5 detection stations. The industrial microscope of each station generates 1 high-definition BGA welding point image (resolution 2048x1536, PNG format, original size about 8MB) every 10 seconds. The image needs to be automatically compressed and backed up for welding point quality traceability and production process optimization.

[0065] 1. System deployment

[0066] The user creates 5 "BGA station XX welding point detection" tasks for the 5 detection stations. The target folder is set as the image output path of the microscope of each station (such as "F:\BGA\Station1\ImageOutput"). The backup directory is "G:\BGABackup\StationXX" path of the storage disk of the factory MES system.

[0067] In the trinity module generated by the system, the configuration file parameter setting is: MaxSize=6MB (PNG format 2048x1536 image original size 8MB, greater than 6MB), dayHour=0 (0 o'clock as the date attribution dividing point, i.e. natural day dividing point), first compression quality parameter=90 (welding point image needs high definition, low loss compression), second compression quality parameter=65, cache capacity parameter=150MB; the batch processing script startup priority is set to "low", which is suitable for the high load running scene of the production line MES system.

[0068] The system deploys the batch processing scripts of the 5 tasks to the self-start directory of the detection station control computer, and completes the deployment.

[0069] 2. Running process

[0070] After the control computer is started, the exclusive execution body of the 5 tasks is automatically started. Each execution body creates an exclusive SQLite database and a.lock file. After the detection process namespace is conflict-free, the file monitoring is started. When the station 1 microscope generates 1 BGA welding point image at 14:30, the execution body locates the new file through hash verification, loads it into the 150MB memory cache, and compresses it with the first compression quality parameter 90 because the image size 8MB is greater than MaxSize=6MB. The compressed size is about 2.1MB. The generation time 14:30 belongs to the current natural day. The image information (including welding point coordinates, detection station number, compression parameter, etc.) is recorded in the exclusive database current date index partition, and then written into the backup directory.

[0071] If the execution body of a task appears compression processing timeout (more than 60 seconds), the exception monitoring module immediately generates exception information and sends it to the production line operation terminal and records the log. The operation personnel query the log to find out that the timeout reason is memory cache fragmentation. After restarting the task execution body, the system returns to normal processing. In addition, when it is necessary to suspend the monitoring task of station 3 to update the microscope parameters, the operation personnel input the "task suspension instruction". This instruction only acts on the exclusive execution body of station 3. The execution body stops monitoring and compression and saves the current task progress to the configuration file. After the parameter update is completed, the "task restart instruction" is input. The execution body loads the progress and resumes operation. During this period, the other four tasks are not affected.

[0072] 3. Effect verification

[0073] During the long-term operation of the 5 tasks, the system has no index conflict and process crash problem. The IO performance is improved by 300% compared with the traditional centralized database. The image retrieval time of a single task is shortened from 1.5 seconds of the traditional method to 0.5 seconds, which is convenient for the production line to quickly trace the quality problem of the welding spot. In terms of storage, the original 8MB PNG image is compressed to about 2.1MB. The storage occupancy of a single station per day (8640 images) is reduced from 69.12GB to 18.144GB, saving about 73.7% of storage space, which meets the storage optimization requirements of the invention disclosure. The CPU utilization is improved by 65% compared with the traditional competitive mode. The average CPU occupancy rate of the control computer is reduced from 75% of the traditional method to 28%, which ensures the stable cooperation of the MES system and the detection equipment.

Claims

1. A multi-task picture automatic compression backup method for an industrial scene, characterized in that, The method comprises the following steps: Step 1, receiving an industrial picture monitoring task created by a user, the industrial picture monitoring task corresponding to a target folder of an industrial machine; Step 2, automatically generating a trinity module corresponding to the monitoring task, the trinity module comprising an exclusive executable body, an exclusive configuration file and an exclusive batch script; Step 3, deploying the exclusive batch script to a system self-starting directory; Step 4, starting the exclusive executable body through the exclusive batch script when the system starts; Step 5, loading the exclusive configuration file by the exclusive executable body, and creating an exclusive SQLite database of the monitoring task in a pre-device backup directory; Step 6, initializing a file system monitoring, and monitoring newly added industrial pictures in the target folder in real time; Step 7, performing automatic compression processing on the newly added industrial pictures, and storing the compressed industrial pictures and corresponding index information in the exclusive SQLite database, thereby realizing distributed independent running and data isolation of multiple tasks.

2. The method of claim 1, wherein the method is applied to an industrial scene. In step 2, the generated exclusive batch script is further configured with a task start priority parameter, and when starting the exclusive executable body, the exclusive batch script sets the running priority of the exclusive executable body to be lower than the normal running priority of the system through a system process scheduling interface; the specific level of the running priority is determined according to a CPU resource occupation threshold of the industrial machine, and when the CPU utilization of the industrial machine exceeds a preset resource protection threshold, the exclusive batch script further reduces the running priority of the exclusive executable body, thereby avoiding that the monitoring task and the core production task of the industrial machine occupy CPU resources, and ensuring the stable execution of the production process of the industrial machine.

3. The method of claim 1, wherein, In step 7, before performing the automatic compression processing, a first conflict protection step is further included: performing a directory hash check on the target folder of the newly added industrial pictures, calculating a current directory structure hash value of the target folder by using a SHA-256 hash algorithm, and comparing the hash value with a historical hash value obtained by last time checking; If the two hash values are consistent, it is determined that there is no new effective file in the target folder, and the subsequent compression processing step is not performed; if the two hash values are inconsistent, the specific path of the new file is located through a hash difference value, and only the new file is subjected to the subsequent compression processing step, thereby avoiding the problems of resource waste and data redundancy caused by repeated processing.

4. The method of claim 3, wherein, The conflict protection further includes a second conflict protection step: creating an exclusive.lock file corresponding to the monitoring task under the pre-device backup directory immediately after the exclusive executable body is started, and the exclusive.lock file is used to mark the running state of the monitoring task; When detecting that multiple monitoring tasks are associated with the same target folder, whether there is a task running conflict is judged by detecting whether the exclusive.lock files corresponding to the monitoring tasks exist; if it is detected that there is an exclusive.lock file of a running task, the execution of the current monitoring task is suspended, the state of the exclusive.lock file is detected again every preset time interval, and until the running task completes processing and releases the exclusive.lock file, the compression backup process of the current monitoring task is started.

5. The method of claim 4, wherein the method is applied to an industrial scene. The conflict protection further includes a third guarantee step: a unique process namespace identifier is assigned to each monitoring task-specific executable, which corresponds to the name of the monitoring task; before the file system monitoring is initialized in step 6, the specific executable first detects whether there is a running process in the current system that duplicates the process namespace identifier of the specific executable; if a duplicate process is detected, the system process management interface is used to terminate the duplicate process, and then the file monitoring process of the specific executable is started; if no duplicate process is detected, the file monitoring process is directly started, thereby achieving complete isolation of different monitoring tasks at the process level.

6. The method of claim 1, wherein, In step 7, the automatic compression process further includes a differentiated compression step based on a preset MaxSize parameter: the MaxSize parameter is preset in the specific configuration file, which is a size demarcation value for dividing industrial pictures; when the size of the newly added industrial picture file monitored is greater than the MaxSize parameter, the first compression quality parameter is used to compress the industrial picture, the first compression quality parameter being a preset low compression loss parameter suitable for large pictures, to ensure that the compressed large picture still meets the clarity requirement of industrial detection; When the size of the newly added industrial picture file is less than or equal to the MaxSize parameter, the second compression quality parameter is used to compress the industrial picture, the second compression quality parameter being a preset high compression efficiency parameter suitable for small pictures, to further improve the storage saving rate of small pictures, wherein the value of the first compression quality parameter is higher than that of the second compression quality parameter.

7. The method of claim 1, wherein the method is applied to an industrial scene. In step 7, it further includes a file date attribution division step based on a preset dayHour parameter: the dayHour parameter is preset in the specific configuration file, which is an integer between 1 and 24, used to determine the storage date attribution of the industrial picture; when the generation time of the newly added industrial picture is monitored to be between 0 o'clock of the current date and the time corresponding to the dayHour parameter, the storage date attribution of the industrial picture is determined to be the previous day, and the index partition corresponding to the previous day is created in the specific SQLite database; when the generation time of the newly added industrial picture is between the time corresponding to the dayHour parameter and 24 o'clock of the current date, the storage date attribution of the industrial picture is determined to be the current day, and the index partition corresponding to the current day is created in the specific SQLite database, thereby achieving ordered storage and fast retrieval of industrial pictures by date.

8. The method of claim 1, wherein, It further includes an abnormality warning step: an abnormality monitoring module is integrated in the specific executable, which monitors the running state of the specific executable, the compression processing progress of the industrial picture, and the read-write state of the specific SQLite database in real time; when it is monitored that the specific executable has no response, the compression processing time exceeds the preset time length, or the database read-write fails, the abnormality monitoring module immediately generates abnormality information, which includes the abnormality type, the abnormality occurrence time, the corresponding monitoring task name, and the abnormality occurrence position. The abnormality monitoring module sends the generated abnormality information to a preset contact terminal, and creates a log file locally, and records complete abnormality details in the log file, which is named according to a monitoring task name and a date, so as to facilitate subsequent query and troubleshooting by an operation and maintenance personnel.

9. The method of claim 1, wherein, In step 7, when performing automatic compression processing on the newly added industrial picture, a memory cache optimization step is further included: when the dedicated executable body is started, a dedicated memory cache area for the monitoring task is initialized according to a preset cache capacity parameter in the dedicated configuration file; when the newly added industrial picture is monitored, the file data of the newly added industrial picture is first loaded into the dedicated memory cache area, and then the compression processing operation is completed in the memory cache area; After the compression processing is completed, the compressed industrial picture data is written from the memory cache area to the pre-device directory, and the corresponding index information is written into the dedicated SQLite database; during the compression processing, the dedicated executable body monitors the occupancy rate of the dedicated memory cache area in real time, and when the occupancy rate exceeds a preset cache threshold, the cache space occupied by the picture data that has completed the compression processing is automatically released, so as to avoid memory overflow affecting the normal operation of the monitoring task.

10. The method of claim 1, wherein the method is applied to an industrial scene. Further, a task life cycle independent control step is included: a dedicated life cycle control instruction is configured for each monitoring task, and the life cycle control instruction includes a task start instruction, a task pause instruction, a task restart instruction and a task termination instruction; when a certain monitoring task needs to be operated, the corresponding life cycle control instruction is input, and the instruction only acts on the dedicated executable body of the monitoring task; when the task pause instruction is executed, the dedicated executable body stops the file monitoring and compression processing operation, and saves the current task progress to the dedicated configuration file; when the task restart instruction is executed, the dedicated executable body loads the task progress saved in the dedicated configuration file, and recovers the file monitoring and compression processing operation; when the task termination instruction is executed, the dedicated executable body first deletes the corresponding dedicated.lock file, and then releases the CPU resources and memory resources occupied by the task, so as to realize independent operation and maintenance of a single monitoring task, without affecting the operation of other monitoring tasks.

Images