Industrial environment data analysis method, device and system based on AI model

Abstract

The embodiment of this specification discloses an industrial environment data analysis method based on an AI model, the method comprising: obtaining first data to be inspected; analyzing the first data to be inspected through a small model to obtain a first analysis result; obtaining second data to be inspected; and obtaining a final analysis result through a large model analysis based on the first analysis result and the second data to be inspected. According to the technical solution of the present invention, real-time reasoning and improved recall rate are performed through a small model, and false alarm detection is performed through a large model, which can make the final analysis result more accurate, the coordinates of the detected target more accurate, and the confidence rate of the model detection improved. It also greatly shortens the time for reasoning operations using only a large model, improves the effectiveness of the large model application, and ultimately achieves an overall improvement in AI analysis performance.

Description

Technical Field

[0001] This specification relates to the field of data analysis technology, and in particular to an industrial environment data analysis method, device and system based on an AI model. Background Art

[0002] In industrial production, it is necessary to conduct safety monitoring of the industrial environment. Various sensors can be deployed in the production environment to monitor the safety of the production environment by analyzing the images, sounds, temperature and humidity data collected by the sensors.

[0003] In recent years, with the development of AI technology, the use of neural network models to intelligently analyze monitoring data in industrial environments has become a major trend. Before 2023, AI analysis in many industrial fields was mainly based on small AI models, which had low detection accuracy. Starting from 2023, large AI models will gradually enter the industrial field.

[0004] However, large AI models consume more computing resources and take longer to detect. In scenarios with large amounts of data to be inspected, it is difficult to ensure the real-time performance of the detection task, which affects production safety. Especially in mining production, the safety risks caused by missed or delayed inspections are extremely high. Therefore, reliable and more real-time monitoring of industrial environments is an urgent problem to be solved. Summary of the invention

[0005] The present invention provides an industrial environment data analysis method, device and system based on an AI model, which are used to solve the problem of low detection efficiency in the application of large AI models.

[0006] To solve the above technical problems, the present invention is achieved as follows:

[0007] On the one hand, the present invention provides an industrial environment data analysis method based on an AI model, comprising:

[0008] Acquire first data to be inspected;

[0009] Analyze the first to-be-tested data by using a small model to obtain a first analysis result;

[0010] Acquiring second data to be inspected;

[0011] Based on the first analysis result and the second data to be inspected, obtaining a final analysis result through large model analysis;

[0012] The first data to be inspected and the second data to be inspected are collected by a data collection device arranged in the industrial environment to be inspected.

[0013] On the other hand, the present invention provides an industrial environment data analysis device based on an AI model, comprising:

[0014] A first data acquisition module is configured to acquire first to-be-tested data;

[0015] A small model analysis module is configured to analyze the first to-be-tested data through a small model to obtain a first analysis result;

[0016] A second data acquisition module is configured to acquire second to-be-tested data;

[0017] A large model analysis module is configured to obtain a final analysis result through large model analysis based on the first analysis result and the second data to be inspected;

[0018] The first data to be inspected and the second data to be inspected are collected by a data collection device arranged in the industrial environment to be inspected.

[0019] On the other hand, the present invention provides an industrial environment data analysis device based on an AI model, comprising:

[0020] A first data acquisition module is configured to acquire first to-be-tested data;

[0021] A small model analysis module is configured to analyze the first to-be-tested data through a small model to obtain a first analysis result;

[0022] A second data acquisition module is configured to acquire second to-be-tested data;

[0023] A large model analysis module is configured to obtain a final analysis result through large model analysis based on the first analysis result and the second data to be inspected;

[0024] The first data to be inspected and the second data to be inspected are collected by a data collection device arranged in the industrial environment to be inspected.

[0025] On the other hand, the present invention provides an industrial environment data analysis system based on an AI model, comprising:

[0026] Small model unit, large model unit and microprocessor unit;

[0027] The microprocessor unit is used to allocate computing resources so that the small model unit and the large model unit execute the aforementioned method to obtain a final analysis result.

[0028] On the other hand, the present invention also provides an industrial environment detection system based on an AI model, comprising:

[0029] Image acquisition equipment, environmental sensors, and the industrial environment data analysis system based on the AI ​​model as mentioned above.

[0030] On the other hand, the present invention also provides an electronic device for industrial environment data analysis based on an AI model, the electronic device comprising at least one processor; and

[0031] a memory communicatively coupled to the at least one processor;

[0032] in,

[0033] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0034] On the other hand, the present invention further provides a computer-readable storage medium, which stores executable instructions. When the instructions are executed by a processor, the processor implements the method described in the present invention.

[0035] The embodiments of the present invention adopt the above-mentioned technical solutions to achieve the following beneficial effects: through the mode of combining large and small models, the problems of insufficient precision of small models and large resource consumption and low effectiveness of large models can be well solved. When applied, real-time reasoning and recall rate improvement can be achieved through small models, and false alarm detection can be performed through large models to make the final analysis results more accurate, the coordinates of the detected targets more accurate, and the accuracy of model detection can be improved. The false detection results of small models can also be corrected to positive detection results, thereby improving the recall rate. The application of the technical solution of the present invention not only greatly shortens the reasoning operation time of the AI ​​model and improves the effectiveness of model application, but also completes the detection task with higher precision and recall rate, solves the contradiction between the precision and recall rate of a single model, and realizes the overall improvement of AI analysis performance. The technical solution of the present invention is applied in the detection of industrial environments, which can effectively reduce the occurrence of missed detections or false detections, the detection results are stable and reliable, and the real-time processing of a large number of video images and environmental monitoring data to be inspected is higher, thereby ensuring the safety of industrial production.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1Schematic diagram of an application scenario of an industrial environment data analysis method based on an AI model in an embodiment of the present invention;

[0039] Figure 2 Flow chart of an industrial environment data analysis method based on an AI model according to an embodiment of the present invention;

[0040] Figure 3 An operation flow chart for implementing the data analysis method in an embodiment of the present invention;

[0041] Figure 4 is a structural block diagram of a data analysis device according to an embodiment of the present invention;

[0042] Figure 5 is a structural block diagram of a data analysis system according to an embodiment of the present invention;

[0043] Figure 6 4 is a structural block diagram of an industrial environment detection system based on an AI model according to an embodiment of the present invention;

[0044] FIG7 is an example of a detection result of a small model and a large model according to an embodiment of the present invention;

[0045] FIG. 8 is another example of detection results of the small model and the large model in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the drawings.

[0047] In this specification, it should be understood that terms such as "including" or "having" are intended to indicate the existence of features, numbers, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts or combinations thereof exist or are added. It should also be noted that the embodiments in this specification and the features in the embodiments may be combined with each other if there is no conflict.

[0048] It should be noted that the method of the present invention is applicable to monitoring applications in most industrial production environments, including mining areas, electricity, logistics, transportation, manufacturing production environments, etc. This specification will take mining production as an example to illustrate and give specific implementation methods.

[0049] In recent years, AI intelligent analysis has played an increasingly important role in smart mines, and the country's requirements for the advancement of AI construction have also become higher and higher. The country has attached increasing importance to the AI ​​construction of smart mines. On February 25, 2020, eight ministries and commissions of the state jointly issued a document requiring all coal mines to be intelligent by 2035; on October 23, 2022, it issued Document No. 128 of the Mine Supervision, requiring the completion of AI video monitoring and behavioral intelligent analysis and identification of all coal mines and 2,400 non-coal mines by 2026. Before 2023, AI analysis of smart mines was mainly based on small AI models (hereinafter referred to as small models). Starting from 2023, with the upgrade of hardware computing power, large AI models (hereinafter referred to as large models) have gradually come onto the stage of history.

[0050] The small model and the large model in the present invention are both deep learning models, which can be constructed through various existing mainstream frameworks and algorithms. The present invention does not limit the specific construction method and training process of the model. The main difference between the small model and the large model in the present invention is the input parameter amount of the model. The input parameter amount of the small model is generally thousands, at most tens of thousands, with the advantages of light weight, fast calculation and low power consumption. The small model is difficult to overcome the low precision (Precision, also known as accuracy) due to the small number of parameters. Even if a large amount of data is used to train the small model, the improvement of its precision is very limited. The number of parameters of the large model is usually between millions and billions. For example, the Pangu large model reaches 100 billion, and chatGPT4 reaches as much as 100 trillion, and its parameter amount is still continuing to surge. This makes the large model have a better reasoning basis than the small model. The upper limit of the precision of the large model is much higher than that of the small model, and the accuracy can be quickly improved using a relatively small number of samples.

[0051] Although large models have the advantage of high accuracy over small models, they require a lot of computing resources to complete training due to their huge number of parameters. After training, large models also require relatively high computing resources to complete inference calculations. In other words, the speed of obtaining inference results using large models is much slower than that of small models. Although small models can achieve fast and lightweight calculations, their accuracy is not ideal.

[0052] On the other hand, in practical applications, users always hope that the model's precision and recall (also known as the recall rate) are as high as possible. However, for a model, this is contradictory: if you want a high precision, the model's confidence threshold must be increased; if you want a high recall, the model's confidence threshold must be lowered.

[0053] In other words, whether it is a large model or a small model, it is difficult to balance the contradiction between recall rate and precision rate when applied independently. Although the large model has the advantage of high precision, its high resource consumption and low effectiveness will also affect its application and promotion.

[0054] To this end, the present invention proposes a mode of combining large and small models for application, which can effectively solve the problems of low precision of small models and low effectiveness of large models. By using a small model to perform real-time reasoning and improve the recall rate, and by using a large model combined with environmental parameters to perform secondary detection on the analysis results of the small model, the precision and recall rates can be improved, effectively solving the contradiction between the recall rate and the precision rate in the detection task. The method of using a small model for initial screening and a large model for re-examination greatly reduces the amount of data to be tested that needs to be calculated by the large model, greatly shortens the time for AI analysis and reasoning, improves the effectiveness of the large model application, and achieves an overall improvement in AI analysis performance.

[0055] Specifically, an embodiment of the present specification provides an industrial environment data analysis method based on an AI model, the method comprising: obtaining first data to be inspected; analyzing the first data to be inspected through a small model unit to obtain a first analysis result; obtaining second data to be inspected; based on the first analysis result and the second data to be inspected, obtaining a final analysis result through a large model analysis; wherein the first data to be inspected and the second data to be inspected are collected by a data acquisition device set in the industrial environment to be inspected.

[0056] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0057] Figure 1 The following is a schematic diagram of an application scenario of the data analysis method in the embodiment of this specification. It should be noted that: Figure 1 What is shown is merely an example of an application scenario of the data analysis method of the embodiment of this specification to help those skilled in the art understand the technical content of the present invention, but it does not mean that the method of the present invention cannot be applied to other scenarios.

[0058] like Figure 1 As shown, the scene is a typical scene in a mining area, which includes various data acquisition equipment 110, industrial ring network 120, and AI analysis system 130.

[0059] The data acquisition device 110 includes various sensors deployed in the industrial environment, including cameras, methane sensors, sound sensors, and other sensors, which are used to collect video images and various environmental data in the industrial environment, and upload the collected data through the industrial ring network 120.

[0060] Industrial ring network 120 connects various systems and equipment in the production environment to achieve real-time monitoring and management of the production process. Industrial ring network 120 usually uses industrial-grade Ethernet switches and routers, shielded twisted pair cables or optical fiber cables that meet industrial standards, and can support high-speed data transmission and large-scale equipment connection to ensure the stability and reliability of data transmission. The ring network equipment under the mine also needs to meet the explosion-proof requirements.

[0061] The AI ​​analysis system 130 is used to analyze and reason the video images and environmental data reported by the data acquisition device 110, detect abnormal conditions in the environment and issue an alarm. The AI ​​analysis system 130 includes various hardware resources and software resources required for AI model operation.

[0062] The scene may also include a monitoring station 140, which may be a general device such as a personal computer, a smart phone, or a dedicated device. The monitoring station 140 is used to receive the detection results of the AI ​​analysis system and perform corresponding control operations by connecting to the production management system.

[0063] Combine the following Figure 2 An embodiment of the industrial environment data analysis method based on the AI ​​model in the present invention is described.

[0064] Figure 2 The flowchart of the data analysis method according to the embodiment of the present invention is schematically shown.

[0065] like Figure 2 As shown, the method includes operations S210 to S240.

[0066] In operation S210, first to-be-checked data is acquired.

[0067] In operation S220, the first to-be-tested data is analyzed by the small model unit to obtain a first analysis result.

[0068] In operation S230, second data to be inspected is acquired, wherein the first data to be inspected and the second data to be inspected are acquired by a data acquisition device disposed in the industrial environment to be inspected.

[0069] In operation S240 , a final analysis result is obtained through large model analysis based on the first analysis result and the second to-be-tested data.

[0070] In an embodiment of the present invention, the first data to be inspected and the second data to be inspected are acquired by a data acquisition device set in the industrial environment to be inspected. In some embodiments, the first data to be inspected includes image data collected by an image acquisition device, and the second data to be inspected includes sound data collected by a sound sensor device. The second data to be inspected also includes data collected by a smoke and fire sensor. For example, in order to monitor the real-time situation of the mining face under a coal mine, it is necessary to detect the state of the gun head of the tunnel boring machine, such as whether it falls to the ground, whether it hits coal gangue, etc. At this time, the first detection data can be a video image, and the second detection data can be sound. For another example, in order to determine the safety status of the miners, the behavior and posture of the miners can be detected. At this time, the first detection data can be a video image, and the second detection data can be sound. For another example, in order to detect whether there is smoke and fire in the underground environment, the first detection data can be a video image, and the second detection data can be smoke and fire sensor data, or methane concentration detection data, etc.

[0071] In the embodiments of the present invention, both the small model and the large model are deep learning models, which can be constructed by various existing mainstream frameworks and algorithms. The present invention does not limit the specific construction method and training process of the model. The main difference between the small model and the large model in the embodiments of the present invention is the number of input parameters of the model. The number of input parameters of the small model is generally thousands, at most tens of thousands, and the number of parameters of the large model is usually between millions and billions, or even more.

[0072] In some embodiments, the small model includes one or more target detection models of simple tasks to perform specific detection tasks, such as detecting abnormalities of the cannon head from video images, whether there is a picture of the cannon head falling or hitting coal gangue, or detecting whether there is a miner not wearing a helmet or a miner falling to the ground from the video image. In this embodiment, due to the single task and small number of parameters of the small model, the time to complete the analysis and reasoning can generally reach the millisecond level, and the typical value is 20 milliseconds to complete the analysis of a frame of image. If it is necessary to support multiple detection tasks, multiple small models can be used to complete them. At this time, the number of small models is m, and at least two of the small models are set to perform different detection tasks. For example, one small model is used to detect the state of the cannon head, and the other small model is used to detect the state of the miner.

[0073] In other implementations, in order to support multi-task detection, the small model can also be one or more moderately sized multi-task detection models that can complete multiple detection tasks. For example, a small model is used to detect the state of the cannon head and the state of the mine car. The analysis and reasoning efficiency of the small model performing multi-task detection may be reduced accordingly, and it can be comprehensively considered according to the needs and actual situation when applied.

[0074] In an embodiment of the present invention, the first analysis result is data of a preset abnormal situation detected by the small model. For example, the preset abnormal situation includes the blast head hitting the coal gangue. If the small model detects the abnormal situation, the detected video image frame is output as the first analysis result. If the preset abnormal situation includes the miner falling to the ground, the small model outputs the image frame of the miner falling to the ground as the first analysis result. In other words, the small model preliminarily screens a large number of video images, selects suspicious image frames from them, and then hands them over to the large model for processing.

[0075] Furthermore, in order to reduce the missed detection rate or increase the recall rate, the confidence threshold of the small model can be lowered so that the small model can detect more suspicious anomalies. For example, if the confidence threshold of the small model is lowered from 0.9 to 0.6 or 0.4, the small model can detect most of the anomalies. Of course, when the recall rate is increased, the precision rate will decrease, and the number of false positives in the first analysis results will increase. However, these false positives can be rechecked through the secondary detection of the large model, thereby ensuring that the overall recall rate and precision rate of the AI ​​system are within the ideal range.

[0076] In the embodiment of the present invention, the large model reads the required second detection data based on the first analysis result of the small model, and further analyzes and infers the first analysis result to determine whether it is a preset abnormal situation. Since the parameter quantity of the large model unit is hundreds of billions or even trillions, and various environmental data must be analyzed, the speed of its analysis and inference is in seconds, and the typical value is that 1 frame of data can be processed in 1 second. In the embodiment of the present invention, since the first analysis result only includes the suspicious data screened out by the small model, the computing load of the large model is greatly reduced, and the computing efficiency of the AI ​​system is improved as a whole.

[0077] Furthermore, if the small model completes multiple detection tasks, the first analysis result may include the detection results of multiple detection tasks. For example, the small model detects both the abnormality of the cannon head and the abnormality of the miner, and the large model needs to recheck the two abnormal images in turn. At this time, the detection tasks can be performed in order of priority. That is, if the first analysis result includes the detection results of at least two detection tasks, the corresponding data is selected from the second data to be inspected according to the priority order of the detection tasks, and the final analysis results corresponding to the detection tasks are obtained in turn through the large model analysis. For example: the abnormality of the cannon head has a higher priority than the abnormality of the mine car, and the miner falling to the ground has a higher priority than the miner not wearing a helmet. The large model needs to read the data of the sound sensor from the second data to be inspected to recheck the abnormality of the cannon head, and so on.

[0078] In some implementations, there may be multiple large models, which can simultaneously review the test results of multiple small models. In particular, when a small model needs to perform multiple test tasks, multiple large models can provide better support for the review of the test tasks. That is, the system can design m small models and n large models according to demand, without loss of generality m>n. At this time, the requirements for computing resources are high, and effective resource scheduling is also required. Scheduling large models for secondary testing according to the priority order of the test tasks is still a better solution.

[0079] According to the technical solution of the embodiment of the present invention, real-time reasoning is performed through a small model, and false alarm detection is performed through a large model, so that the final analysis result is more precise, the coordinates of the detected target are more accurate, and the accuracy of the detection result is improved. The false detection result can also be corrected to a positive detection result, which improves the recall rate of the system. It also greatly shortens the time for reasoning operations using only a large model, improves the effectiveness of the large model application, and ultimately achieves an overall improvement in AI analysis performance.

[0080] Combine the following Figure 3 A specific embodiment of the technical solution of the present invention is described.

[0081] Figure 3 The following is a flowchart of the data analysis method according to the embodiment of the present invention. Figure 3 As shown, the operation process is as follows:

[0082] 1) Set model parameters: including small model parameters and large model parameters. The parameters here mainly refer to some parameter quantities required by the AI ​​model, such as weights and bias values. Especially for small models, the recall rate is appropriately increased, while the improvement of accuracy depends on re-examination of the large model.

[0083] 2) Small model analysis and detection: Without loss of generality, the analysis and detection of the small model is mainly based on the video data of the camera. If an abnormal situation is detected, for example, in the tunneling working face of the coal mine, the gun head of the tunneling machine is detected to have fallen to the ground or the gun head has hit the coal gangue, then the video image of this frame (i.e., the first analysis result) is sent to the large model.

[0084] 3) The large model obtains sensor data: For example, at the tunneling working face in a coal mine, the small model detects that the gun head of the tunneling machine has hit coal gangue. At this time, the large model also needs to obtain real-time data from the sound sensor.

[0085] 4) Large model analysis and detection: Combined with the detection results of the small model and the sensor data, the large model comprehensively calculates and obtains the final analysis results. For example, the large model rechecks the video images detected by the small model, and combines the data sent back by the sound sensor to determine whether the gun head of the tunnel boring machine has fallen off. If the large model also determines that the gun head has fallen off through detection, the final result will be reported so that the monitoring center can take further measures, such as stopping tunneling to avoid damage to the gun head.

[0086] During the analysis and detection process of the large model, on the one hand, the large model can correct the coordinate position of the target detected by the small model, such as Figure 7a The following is the detection result of the small model. The number 0.7 is the confidence rate that the target in the box is the correct target. Figure 7b The final result shown in Figure 1 shows that the target is more accurately labeled and the confidence rate reaches 0.9. On the other hand, the large model can also correct the false positives of the small model into true positives, thus improving the recall rate of the system. Figure 8a As shown in the figure, the position of the cannon head identified by the small model is wrong (the actual cannon head is in the center), Figure 8b This is the position of the cannon head after the large model is corrected. At this time, the cannon head marking is correct. Obviously, after the large model analysis, the system can give more accurate recognition results and alarm information.

[0087] Compared with the current simple large model analysis system, the detection speed is significantly improved after applying the technical solution of the present invention. This is because the current simple large model analysis system needs to detect all video frames, and due to its large scale, the operation time is in seconds, and the typical value is 1 second to process 1 frame of data. In the embodiment of the technical solution of the present invention, the small model completes a simple target detection task, and the operation time is in milliseconds. The typical value is 20 milliseconds to complete the analysis of a frame of image, and the large model only needs to recheck the video frames detected by the small model. A typical set of data is: the small model alarms 200 times within 24 hours, and the large model only needs to process 200 frames within 24 hours to complete the detection task, which greatly reduces the computational load of the AI ​​analysis system. When there is a large amount of video data to be processed in the monitored environment, the detection time of abnormal situations can also be significantly reduced, thereby improving real-time performance.

[0088] Based on the same inventive concept, this specification also provides an industrial environment data analysis device 400 based on an AI model. Figure 4 The data analysis device 400 will be described.

[0089] Figure 4The block diagram of the industrial environment data analysis device 400 based on the AI ​​model implementing the embodiment of the present invention is schematically shown. The data analysis device 400 can be implemented as part or all of the electronic device through software, hardware or a combination of both.

[0090] like Figure 4 As shown, the data analysis device 400 includes a first data acquisition module 410, a small model analysis module 420, a second data acquisition module 430, and a large model analysis module 440. The data analysis module 400 can execute the various methods described above.

[0091] The first data acquisition module 410 is configured to acquire first data to be inspected.

[0092] The small model analysis module 420 is configured to analyze the first to-be-tested data by using a small model to obtain a first analysis result.

[0093] The second data acquisition module 430 is configured to acquire second data to be inspected, wherein the first data to be inspected and the second data to be inspected are acquired by a data acquisition device disposed in the industrial environment to be inspected.

[0094] The large model analysis module 440 is configured to obtain a final analysis result through large model analysis based on the first analysis result and the second data to be inspected.

[0095] The data analysis device according to the embodiment of the present invention can solve the problems of insufficient accuracy of small models and high resource consumption and low effectiveness of large models by combining large and small models, effectively balance the contradiction between recall rate and precision rate in detection tasks, and achieve an overall improvement in AI analysis performance.

[0096] Based on the same inventive concept, this specification also provides an industrial environment data analysis system 500 based on an AI model. Figure 5 The data analysis system 500 will be described.

[0097] Figure 5 The block diagram of the AI ​​model-based industrial environment data analysis system 500 for implementing the embodiment of the present invention is schematically shown. The data analysis system 500 can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0098] like Figure 5 As shown, the data analysis system 500 includes a microprocessor unit 510 , a small model unit 520 and a large model unit 530 .

[0099] The microprocessor unit 510 is used to allocate computing resources to enable the small model unit 520 and the large model unit 530 to execute various data analysis methods as described above to obtain the final analysis results.

[0100] Furthermore, the data analysis system 500 may also include a storage unit 540 and an AI batch computing resource unit 550 .

[0101] The storage unit 540 is used to store the data to be inspected, including video images collected and transmitted by the camera, sounds collected and transmitted by the sensor, and other environmental data. The resources of the storage unit are uniformly scheduled by the microprocessor.

[0102] The AI ​​batch computing resource unit 550 is used to provide batch computing resources to the small model unit 520 or the large model unit 530 under the scheduling of the microprocessor unit 510.

[0103] AI batch computing resources are hardware resources required for AI models to perform batch AI computing. In practice, the GPUs (graphics cards) currently used, such as NVIDIA's T4 graphics card and Huawei's A300I accelerator card, all belong to this type of resource. Its characteristics are the ability to input data in batches, complete calculations, and return calculation results. For example, when a small model is analyzing and calculating, the data in the memory will be organized into vectors in batches (patch), and the whole will be poured into this unit for inference calculation. Similar batch operations are also used for large model analysis and calculation.

[0104] Further, in the data analysis system according to the embodiment of the present invention, the number of small model units is m, and at least two of the small model units are configured to perform different detection tasks. In this case, the microprocessor unit is also used to control the large model unit to perform logical operations according to the priority order of the detection tasks. By performing operations according to the priority order of the detection tasks, actual business requirements can be met.

[0105] The data analysis system according to the embodiment of the present invention can solve the problem of insufficient accuracy of small models and low effectiveness of large models through the mode of combining large and small models. Through the resource scheduling of the microprocessor unit, it can effectively support multiple small models in parallel for multiple detection tasks. Then, through the orderly re-inspection of the large model, it can not only effectively balance the contradiction between recall rate and precision rate in the detection task, but also reduce the computing load of the large model and improve the analysis efficiency. The overall performance of the AI ​​analysis system can be better and the application method can be more flexible, suitable for application in various industrial environments.

[0106] Based on the same inventive concept, this specification also provides an AI model-based detection system 600 for an industrial environment.

[0107] like Figure 6As shown, the detection system includes an image acquisition device 610, an environmental sensor 620, and the industrial environment data analysis system 500 based on the AI ​​model as described above. The image acquisition device 610 and the environmental sensor 620 are set in the industrial environment to be detected, such as underground coal mines, power generation stations, manufacturing workshops, etc. The required environmental data is collected in real time, and the data is uploaded to the data analysis system 500 through a communication network such as an industrial ring network.

[0108] According to the scheme of the embodiment of the present invention, data is collected by various sensors in the industrial environment, and the mode of combining large and small models is used to solve the problems of insufficient accuracy of small models and low effectiveness of large models due to large resource consumption, and balance the contradiction between recall rate and precision rate in the detection task. Real-time reasoning and improved recall rate are performed by small models, and false alarm detection is performed by large models to make the final analysis results more accurate, the coordinates of the detected target more accurate, and the confidence rate of model detection is improved. The false detection results of the small model can also be corrected to positive detection results, the recall rate is improved, and the reasoning operation time is greatly shortened. The effectiveness of model application is improved, and the overall improvement of AI analysis performance is achieved. When the technical solution of the present invention is applied in industrial environment monitoring, it effectively reduces the occurrence of missed detections or false detections, the detection results are stable and reliable, and the real-time processing of a large number of video images and environmental monitoring data to be inspected is higher, ensuring the safety of industrial production.

[0109] This specification also provides an electronic device for industrial environment data analysis based on an AI model, the electronic device comprising at least one processor; and,

[0110] a memory communicatively coupled to the at least one processor;

[0111] in,

[0112] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0113] This specification also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores executable instructions, which, when executed by a processor, enable the processor to implement the industrial environment data analysis method based on an AI model described in this specification.

[0114] The above describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0116] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0117] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0118] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0119] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0120] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0121] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0122] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data optimization device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data optimization device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data optimization device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data optimization device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0127] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0129] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0130] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0131] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An industrial environment data analysis method based on AI model, characterized in that: The method comprises: Acquire first data to be inspected; Analyzing the first data to be inspected by means of a small model to obtain a first analysis result, wherein the number of the small models is m, and at least two of the small models are configured to perform different inspection tasks, and the first analysis result is data in which a preset abnormal situation is detected; Acquiring second data to be inspected; Based on the first analysis result and the second data to be inspected, obtaining a final analysis result through large model analysis, including: if the first analysis result includes detection results of at least two detection tasks, selecting corresponding data from the second data to be inspected according to the priority order of the detection tasks, and obtaining final analysis results corresponding to the detection tasks through large model analysis in sequence; The first data to be inspected and the second data to be inspected are collected by a data collection device arranged in the industrial environment to be inspected.

2. The method according to claim 1, characterized in that The first data to be inspected includes image data acquired by an image acquisition device.

3. An industrial environment data analysis device based on an AI model, characterized in that: include: A first data acquisition module is configured to acquire first to-be-tested data; a small model analysis module, configured to analyze the first to-be-tested data through a small model to obtain a first analysis result, wherein the number of the small models is m, and at least two of the small models are configured to perform different detection tasks, and the first analysis result is data in which a preset abnormal situation is detected; A second data acquisition module is configured to acquire second to-be-tested data; The large model analysis module is configured to obtain a final analysis result through large model analysis based on the first analysis result and the second data to be inspected, including: if the first analysis result includes the detection results of at least two detection tasks, then according to the priority order of the detection tasks, select corresponding data from the second data to be inspected, and obtain the final analysis results corresponding to the detection tasks through large model analysis in sequence; The first data to be inspected and the second data to be inspected are collected by a data collection device arranged in the industrial environment to be inspected.

4. An industrial environment data analysis system based on AI model, characterized in that: include: A small model unit, a large model unit and a microprocessor unit, wherein the number of the small model units is m, and at least two of the small model units are configured to perform different detection tasks; In which, the microprocessor unit is used to allocate computing resources so that the small model unit and the large model unit execute the method as described in any one of claims 1 to 2 to obtain the final analysis result; the microprocessor unit is also used to control the large model unit to perform logical operations according to the priority order of the detection tasks.

5. The system according to claim 4, characterized in that Also includes: A storage unit, used for storing data to be inspected; An AI batch computing resource unit is used to provide batch computing resources to the small model unit or the large model unit under the scheduling of the microprocessor unit.

6. An industrial environment detection system based on AI model, characterized in that: include: An image acquisition device, an environmental sensor, and an industrial environment data analysis system based on an AI model as described in any one of claims 4 to 5.

Images