Inspection equipment and method
Through preset detailed inspection plans and real-time control, multiple algorithm models are flexible scheduling and inspection analysis in multiple point scenarios, solving the problem of insufficient efficiency and stability in the existing technology, and improving the inspection Flexibility and accuracy reduce labor costs.
Patent Information
- Application Number
- CN202410950375.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-16
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-16
AI Technical Summary
The existing artificial intelligence integrated inspection superbrain cannot adapt to changes in time during multi-algorithm models and multi-point scenario inspections, and cannot flexibly schedule multiple algorithms for analysis, resulting in insufficient efficiency and stability.
Provide a patrol method and equipment, and realize dynamics by presetting detailed patrol plans, including execution periods, physical analysis units, patrol items and their information, periodically detecting and executing, and real-time control of image acquisition equipment and algorithm models. Adjust and optimize, automatically terminate and switch inspection items to ensure that they are started and executed on time.
It improves the flexibility and pertinence of inspections, reduces human intervention, improves accuracy and efficiency, avoids resource waste and redundancy, ensures the stable operation of inspection equipment, and reduces labor costs and resource occupation.
Smart Images

Figure CN118509710B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of patrol inspection task scheduling, and in particular to a patrol inspection device and method. Background Art
[0002] At present, AI-integrated inspection super-brain products have gradually become a rising star in the market, and their positioning is mainly for market segments such as park inspection and safe production. Due to their specific application scenarios and needs, these markets have increasing requirements for AI technology. At present, AI-integrated inspection super-brain can only run fixed algorithms for a long time to analyze fixed channel videos. This means that when the scene or demand changes and the AI-integrated inspection super-brain is required to support running multiple algorithms in a certain period of time for multi-scene inspections, the AI-integrated inspection super-brain may not be able to adapt in time and make corresponding adjustments. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide an inspection device and method to achieve flexible scheduling inspection analysis using multiple algorithm models in multiple point scenarios within a time period, reduce labor costs, and improve the efficiency and stability of inspection work. The specific technical solution is as follows:
[0004] In a first aspect, an embodiment of the present application provides an inspection method, which is applied to an inspection device including a plurality of device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having a plurality of shooting points;
[0005] The method comprises:
[0006] Acquire and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied when executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: execution order, execution duration, analysis mode and algorithm model to be applied when executing the inspection item, and a shooting point position at which the first image acquisition device needs to be when executing the inspection item;
[0007] Periodically detecting whether the first execution period has arrived, and if it has arrived, reading the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item, and initially the target inspection item is the inspection item with the first execution order;
[0008] Control the first physical analysis unit to load the read algorithm model, and control the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data through the first physical analysis unit after the algorithm model is loaded to obtain the inspection result;
[0009] Periodically obtain the duration of the target inspection item that the first physical analysis unit has executed. If the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, control the first physical analysis unit to terminate the execution of the target inspection item, and use the inspection item whose execution order is next to the target inspection item as the new target inspection item, and return to the step of executing the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item;
[0010] Until the first execution period ends, the first physical analysis unit is controlled to terminate execution of the target inspection item.
[0011] In a possible implementation, the method further includes:
[0012] If the first image acquisition device is detected to switch a point during the process of the first physical analysis unit executing the target inspection item, the first physical analysis unit is controlled to suspend execution of the target inspection item;
[0013] If the first image acquisition device is switched back to the read shooting point, the first physical analysis unit is controlled to continue to execute the target inspection item.
[0014] In a possible implementation manner, at least one virtual analysis unit runs in each of the physical analysis units, and the method further includes:
[0015] For each inspection item in the first inspection plan, determine the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the first number, wherein the number of virtual analysis units required to be occupied by the inspection item is positively correlated with the algorithm model complexity of the algorithm model included in the inspection item information of the inspection item, and the number of virtual analysis units required to be occupied by the inspection item including an analysis mode of a real-time analysis mode is higher than the number of virtual analysis units required to be occupied by the inspection item including an analysis mode of a screenshot analysis mode;
[0016] Determine, in the first physical analysis unit, the first number of virtual analysis units to which no inspection plan is allocated during the first execution period as first virtual analysis units, and allocate the first inspection plan to all the first virtual analysis units;
[0017] The controlling the first physical analysis unit to load the read algorithm model and controlling the first image acquisition device to switch to the read shooting point position includes:
[0018] Controlling all the first virtual analysis units in the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point;
[0019] The first physical analysis unit after loading the algorithm model analyzes the sampled image data to obtain an inspection result, including:
[0020] All the first virtual analysis units in the first physical analysis unit after loading the algorithm model analyze the sampled image data to obtain an inspection result.
[0021] In a possible implementation, the method further includes:
[0022] The obtaining and storing a first inspection plan input by a user for the first image acquisition device includes:
[0023] Acquire a first inspection plan input by a user for the first image acquisition device, and search for all second inspection plans including the first physical analysis unit in the stored inspection plans;
[0024] For all sub-periods in the first execution period, respectively, search for all inspection items executed in the sub-periods in all the second inspection plans, and determine the sum of the numbers of virtual analysis units required to be occupied by all the found inspection items as the second number of the sub-periods;
[0025] Determine the maximum value of the second number of all sub-periods as the third number; and for each inspection item in the first inspection plan, determine the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the fourth number;
[0026] If the sum of the third number and the fourth number is not greater than the total number of virtual analysis units in the first physical analysis unit, storing the first inspection plan;
[0027] The method further comprises:
[0028] If the sum of the third number and the fourth number is greater than the total number of virtual analysis units in the first physical analysis unit, refusing to store the first inspection plan;
[0029] Among them, the number of virtual analysis units required for the inspection item is positively correlated with the algorithm model complexity of the algorithm model included in the inspection item information of the inspection item, and the number of virtual analysis units required for the inspection item whose analysis mode is the real-time analysis mode is higher than the number of virtual analysis units required for the inspection item whose analysis mode is the screenshot analysis mode.
[0030] In a possible implementation manner, the inspection device runs a thread for each physical analysis unit; and the method further includes:
[0031] Periodically waking up a first thread running on the first physical analysis unit to obtain the system time of the inspection device through the first thread;
[0032] and determining, through the first thread, whether a difference between the start time of the first execution period and the system time is within a preset difference range, and if so, determining that the first execution period has arrived;
[0033] The step of sampling the image data acquired by the first image acquisition device after the switching point according to the read analysis mode includes:
[0034] If the analysis mode read is the real-time analysis mode, the image data collected by the first image acquisition device after the switching point is sampled in real time; if the analysis mode read is the snapshot analysis mode, the image data collected by the first image acquisition device after the switching point is sampled periodically according to the snapshot interval indicated by the snapshot analysis mode.
[0035] In a possible implementation manner, the inspection device further includes a network unit, and the network unit is used to establish a communication connection between the inspection device and an alarm center; and the method further includes:
[0036] In response to the enabling of the proprietary alarm mode, an alarm mode control is displayed; in response to a configuration instruction for the alarm mode control, a first alarm mode indicated by the configuration instruction is acquired and recorded; if the inspection result meets the preset alarm condition and the proprietary alarm mode is enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition and the proprietary alarm mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode;
[0037] In response to the enabling of the superimposed detection frame mode, if the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained and the corresponding image data that meets the alarm condition. an object detection frame; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode, and the alarm message includes the image data on which the inspection result is based; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode, and the alarm message includes the image data on which the inspection result is based and the detection frame of the object that meets the alarm condition.
[0038] In a second aspect, an embodiment of the present application provides an inspection device, the inspection device comprising a plurality of device interfaces and at least one physical analysis unit, the device interface being used to access an image acquisition device, the image acquisition device comprising a first image acquisition device having a plurality of shooting points;
[0039] The inspection device is used to obtain and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied for executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: execution order, execution duration, analysis mode and algorithm model to be applied for executing the inspection item, and shooting point position at which the first image acquisition device needs to be when executing the inspection item; periodically detecting whether the first execution period has arrived, and if so, reading the analysis mode, algorithm model and shooting point position recorded in the inspection item information of the target inspection item, wherein the target inspection item is the inspection item with the first execution order; controlling the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data by the first physical analysis unit after loading the algorithm model to obtain the inspection result; periodically obtain the duration of the first physical analysis unit executing the target inspection item, if the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, control the first physical analysis unit to terminate the execution of the target inspection item, and use the inspection item with the execution order next to the target inspection item as the new target inspection item, and return to the step of executing the analysis mode, algorithm model and shooting point recorded in the inspection item information of the read target inspection item; until the first execution period has ended, control the first physical analysis unit to terminate the execution of the target inspection item;
[0040] The first image acquisition device is used to respond to the control of the inspection device, switch to a desired shooting point, and acquire image data at the shooting point.
[0041] In a third aspect, an embodiment of the present application provides an inspection device, which is applied to an inspection device including multiple device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having multiple shooting points, and the device includes:
[0042] an acquisition module, configured to acquire and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied when executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: an execution sequence, an execution duration, an analysis mode and an algorithm model to be applied when executing the inspection item, and a shooting point position at which the first image acquisition device needs to be when executing the inspection item;
[0043] A detection module, used for periodically detecting whether the first execution period has arrived, and if so, reading the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item, wherein the target inspection item is initially the inspection item with the first execution order;
[0044] An execution module is used to control the first physical analysis unit to load the read algorithm model, and control the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data by the first physical analysis unit after the algorithm model is loaded to obtain the inspection result;
[0045] A patrol module, used for periodically obtaining the duration of execution of the target inspection item by the first physical analysis unit, and if the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, controlling the first physical analysis unit to terminate the execution of the target inspection item, and taking the inspection item whose execution order is next to the target inspection item as the new target inspection item, and returning to the step of executing the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item;
[0046] A termination module is used to control the first physical analysis unit to terminate the execution of the target inspection item until the first execution period ends.
[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned inspection methods is implemented.
[0048] Beneficial effects of the embodiments of the present application:
[0049] The inspection method provided in the embodiment of the present application greatly improves the flexibility and pertinence of the inspection by presetting a detailed inspection plan, including an execution period, a physical analysis unit, inspection items and their inspection item information, and ensures the comprehensiveness and efficiency of the inspection work. Secondly, through the periodic detection and execution mechanism, it ensures that the inspection task can be automatically started on time and executed strictly in accordance with the preset analysis mode, algorithm model and shooting point, effectively reducing human intervention and improving the accuracy of the inspection. Furthermore, by real-time control of the shooting points of the image acquisition device and the loading of the algorithm model of the physical analysis unit, the dynamic adjustment and optimization of the inspection process is realized, further improving the efficiency of the inspection. Finally, it can automatically terminate and switch to the next inspection item according to the execution time of the inspection item, effectively avoiding unnecessary waste of resources and redundancy of the inspection process, avoiding ineffective occupation of resources, and ensuring the stable operation of the inspection equipment. To sum up, this inspection method realizes the flexible scheduling of inspection and analysis in multiple point scenarios within a period of time using multiple algorithm models. This automated and intelligent inspection method reduces manual intervention, reduces labor costs, avoids ineffective occupation of resources, improves the efficiency and stability of inspection work, and provides more efficient inspection solutions for various application scenarios.
[0050] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0052] Figure 1 A schematic diagram of a first process flow of the inspection method provided in an embodiment of the present application;
[0053] Figure 2 A schematic diagram of an inspection scenario provided in an embodiment of the present application;
[0054] Figure 3 A schematic diagram of the first structure of the inspection device provided in the embodiment of the present application;
[0055] Figure 4 A second structural schematic diagram of the inspection device provided in the embodiment of the present application;
[0056] Figure 5 A schematic diagram of a conflict check provided in an embodiment of the present application;
[0057] Figure 6A first schematic diagram of inspection plan allocation provided in an embodiment of the present application;
[0058] Figure 7 A second schematic diagram of inspection plan allocation provided in an embodiment of the present application;
[0059] Figure 8 A third structural schematic diagram of the inspection device provided in the embodiment of the present application;
[0060] Fig. 9 A fourth structural schematic diagram of the inspection device provided in the embodiment of the present application;
[0061] Fig.10 A second flow chart of the inspection method provided in the embodiment of the present application;
[0062] Fig.11 A schematic diagram of the structure of the inspection device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.
[0064] In order to more clearly explain the inspection method provided by this application, the relevant terms involved in this article are explained below:
[0065] Artificial Intelligence Fusion Inspection Super Brain: Used in conjunction with a camera, it is a device that can perform multiple algorithm intelligent analyses on videos / images captured by the camera. In this article, for the sake of convenience, the Artificial Intelligence Fusion Inspection Super Brain is referred to as the Super Brain.
[0066] Physics engine: It is a hardware device or module with high-performance computing capabilities. In this article, it is called a physics analysis unit for the convenience of description.
[0067] Virtual engine: The virtualization unit of the physical engine can load a model algorithm independently for intelligent analysis. In this article, it is called a virtual analysis unit for the convenience of description.
[0068] In the existing technology, Ultrain mainly serves the market segments such as park inspection and safe production. Although it integrates more than ten intelligent algorithms and has the ability to perform intelligent analysis on various scenarios, the current Ultrain system is still limited by the physical analysis unit, which can only run a fixed algorithm model for a long time to analyze the channel video of a fixed channel. Therefore, it cannot fully meet the needs when dealing with multi-algorithm models and multi-point scene inspections.
[0069] Regarding multi-algorithm models and multi-point scene inspections, the main situations encountered in actual applications are as follows:
[0070] Case 1: The same camera may need to apply different algorithm models and analysis modes (such as real-time video analysis, timed capture analysis, and camera analysis) to perform intelligent analysis according to different time periods. For example, in an open kitchen scene, during the day, you may focus on the wearing of white chef hats, white chef uniforms, and white masks, while at night, you may focus on the detection of rodent infestation.
[0071] Case 2: The same dome camera has different preset points. In different time periods, the dome camera is actively switched to the specified preset points to collect image data. Different algorithm models are used to analyze the collected image data. If the dome camera leaves the preset point, the super brain automatically stops the analysis. After the dome camera returns to the preset point, the super brain continues the analysis. For example, in the ecological environment monitoring scenario: the same dome camera rotates to preset point A to detect whether there is garbage on the street, and after a period of time, it rotates to preset point B to detect whether there is a fire on the other side of the street, and after a period of time, it rotates to preset point C to detect whether there are floating objects in the river. These points can be switched for detection in a round-robin manner without the need to continuously analyze all points in real time.
[0072] In order to meet the above market demands, it is necessary to further enhance the multi-algorithm inspection capabilities of Super Brain, so as to realize flexible scheduling and inspection analysis in multiple point scenarios within a period of time using multiple algorithm models, reduce labor costs, and improve the efficiency and stability of inspection work.
[0073] Based on this, the present application provides an inspection method, which is applied to an inspection device, wherein the inspection device includes multiple device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having multiple shooting points, see Figure 1 , Figure 1 A first flow chart of the inspection method provided in the embodiment of the present application, the method comprising:
[0074] S101, obtaining and storing a first inspection plan input by a user for a first image acquisition device, the first inspection plan comprising: a first execution period of the first inspection plan, a first physical analysis unit to be applied to execute the first inspection plan, at least one inspection item, and inspection item information of each inspection item.
[0075] Among them, the inspection item information includes: execution order, execution duration, analysis mode and algorithm model to be applied when executing the inspection item, and the shooting point where the first image acquisition device needs to be when executing the inspection item.
[0076] S102, periodically checking whether the first execution period has arrived, if it has arrived, reading the analysis mode, algorithm model and shooting points recorded in the inspection item information of the target inspection item, initially the target inspection item is the inspection item with the first execution order.
[0077] S103, controlling the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point; and executing the target inspection item in the following manner: sampling the image data acquired by the first image acquisition device after the switching point according to the read analysis mode, and analyzing the sampled image data through the first physical analysis unit after loading the algorithm model to obtain the inspection result.
[0078] S104, periodically obtain the duration of the target inspection item that the first physical analysis unit has executed. If the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, control the first physical analysis unit to terminate the execution of the target inspection item, and use the inspection item that is next in execution order to the target inspection item as the new target inspection item, and return to the step of executing the analysis mode, algorithm model and shooting points recorded in the inspection item information of the target inspection item.
[0079] S105 , until the first execution period ends, controlling the first physical analysis unit to terminate execution of the target inspection item.
[0080] By applying the inspection method provided in the embodiment of the present application, a detailed inspection plan is preset, including an execution period, a physical analysis unit, inspection items and their inspection item information, which greatly improves the flexibility and pertinence of the inspection, and ensures the comprehensiveness and efficiency of the inspection work. Secondly, through the periodic detection and execution mechanism, it is ensured that the inspection task can be automatically started on time, and is strictly executed in accordance with the preset analysis mode, algorithm model and shooting point, which effectively reduces human intervention and improves the accuracy of the inspection. Furthermore, by real-time control of the shooting point of the image acquisition device and the loading of the algorithm model of the physical analysis unit, the dynamic adjustment and optimization of the inspection process is realized, and the efficiency of the inspection is further improved. Finally, it can automatically terminate and switch to the next inspection item according to the execution time of the inspection item, effectively avoiding unnecessary waste of resources and redundancy of the inspection process, avoiding ineffective occupation of resources, and ensuring the stable operation of the inspection equipment. To sum up, this inspection method realizes the flexible scheduling of inspection and analysis in multiple point scenarios within a period of time using multiple algorithm models. This automated and intelligent inspection method reduces manual intervention, reduces labor costs, avoids ineffective occupation of resources, improves the efficiency and stability of inspection work, and provides more efficient inspection solutions for various application scenarios.
[0081] For example, see Figure 2 , Figure 2 A schematic diagram of the inspection scenario provided for the embodiment of the present application, such as the aforementioned ecological environment monitoring scenario, the first image acquisition device needs to rotate to point A to detect whether there is garbage on the street, after a period of time, rotate to point B to detect whether there is a fire on the other side of the street, and after a period of time, rotate to point C to detect whether there are floating objects in the river. Using the inspection method of the present application, the user can input the first inspection plan for the first image acquisition device, the first execution period of the first inspection plan is 9:00-15:00, and the first inspection plan includes 3 inspection items, namely inspection item 1, inspection item 2 and inspection item 3.
[0082] The execution order of inspection item 1 is 1, the execution time is 2 hours, the analysis mode is real-time analysis mode, the algorithm model is the garbage detection algorithm model, and the shooting point is point A, that is, inspection item 1 is to detect whether there is garbage on the street at point A.
[0083] The execution order of inspection item 2 is 2, the execution time is 3 hours, the analysis mode is real-time analysis mode, the algorithm model is the fire detection algorithm model, and the shooting point is point B, that is, inspection item 2 is to detect whether there is a fire on the other side of the street at point B.
[0084] The execution order of inspection item 3 is 3, the execution time is 1 hour, the analysis mode is real-time analysis mode, the algorithm model is the floating object detection algorithm model, and the shooting point is point C, that is, inspection item 3 is to detect whether there are floating objects in the river at point C.
[0085] When executing the above-mentioned first inspection plan, the shooting points of the first image acquisition device are point A-point B-point C in sequence. During the period of 9:00-11:00, the first image acquisition device switches to point A, that is, the visual axis 10 of the first image acquisition device is perpendicular to point A, and the garbage detection algorithm model is used at point A to detect whether there is garbage on the street. During the period of 11:00-14:00, the first image acquisition device switches to point B, that is, the visual axis 20 of the first image acquisition device is perpendicular to point B, and the fire detection algorithm model is used at point B to detect whether there is a fire on the street. During the period of 14:00-15:00, the first image acquisition device switches to point C, that is, the visual axis 30 of the first image acquisition device is perpendicular to point C, and the floating object detection algorithm model is used at point C to detect whether there are floating objects in the river. After the first execution period is reached, the execution of each inspection item in the first inspection plan ends.
[0086] Before describing the inspection method described in S101-S105 above, an exemplary description of the inspection device provided by the present application is given first:
[0087] See also Figure 3 , Figure 3The first structural diagram of the inspection device provided in the embodiment of the present application is shown. The inspection device is integrated in the super brain, and the super brain maintains a communication connection with the server. The inspection device includes multiple device interfaces 100 and at least one physical analysis unit 200. The device interface 100 is used to access the image acquisition device 300. The image acquisition device 300 includes a first image acquisition device 400 with multiple shooting points. For the convenience of description, the figure only shows the situation where the inspection device includes two device interfaces 100, two physical analysis units 200, two image acquisition devices 300, and the image acquisition device 300 only includes one first image acquisition device 400. Among them, the image acquisition device can be a device with image acquisition function such as a ball camera, a network camera, and an infrared camera.
[0088] Steps S101 to S105 are described below respectively:
[0089] In S101, the user can input the first inspection plan for the first image acquisition device in the configuration interface of the super brain web page, or can input the first inspection plan for the first image acquisition device in the configuration interface of the server side, without specific limitation here.
[0090] There may be one or more physical analysis units. In the case where there is one physical analysis unit, in order to improve inspection efficiency, in a possible embodiment, at least one virtual analysis unit may be run on the physical analysis unit. Figure 4 , Figure 4 This is a second structural diagram of the inspection device provided in the embodiment of the present application. At least one virtual analysis unit 500 runs on the physical analysis unit 200. For the convenience of description, the figure only shows the case where only one virtual analysis unit 500 runs on the physical analysis unit 200. Taking the input of the first inspection plan in the configuration interface of the server as an example, when the user inputs the first inspection plan, the various inspection items in the first inspection plan can be flexibly assigned to different virtual analysis units for execution.
[0091] In the case where there are multiple physical analysis units, the user can also specify which physical analysis unit and which virtual analysis unit will execute each inspection item in the first inspection plan. However, in the case where the inspection device includes multiple physical analysis units, the user's workload will be heavy if the user specifies a specific physical analysis unit and a specific virtual analysis unit, and errors may also occur.
[0092] Based on this, in a possible embodiment, taking the input of the first inspection plan in the configuration interface of the server as an example, after the first inspection plan is successfully input, the server sends the first inspection plan to the inspection device. After the inspection device stores the first inspection plan through a storage medium, the inspection device can intelligently and automatically allocate physical analysis units to the first inspection plan, and further automatically specify the most appropriate virtual analysis unit for execution for each inspection item in the plan, thereby reducing the user's burden and improving operation accuracy and the overall efficiency of the system.
[0093] Before the inspection device automatically specifies a physical analysis unit for the first inspection plan and specifies a virtual analysis unit for each inspection item in the plan, in order to improve the inspection efficiency and avoid resource conflicts when executing some of the inspection plans, such as multiple inspection plans requesting to use the same physical analysis unit or shooting point in the same time period, in a possible embodiment, in order to avoid waste of resources and improve the efficiency and success rate of the inspection, after receiving the inspection plan sent by the server, the inspection device can first perform a conflict check before automatically allocating each physical analysis unit and each virtual analysis unit. Only when there is no conflict between the currently received inspection plan and the already stored inspection plan, the currently received inspection plan is stored, and each physical analysis unit and each virtual analysis unit are automatically allocated. The specific verification process is as follows. Step S101 includes:
[0094] S1011, obtaining a first inspection plan input by a user for a first image acquisition device, and searching for all second inspection plans including the first physical analysis unit in the stored inspection plans.
[0095] S1012, for all sub-periods in the first execution period, search for all inspection items executed in the sub-periods in all second inspection plans, and determine the sum of the number of virtual analysis units required for all the found inspection items as the second number of sub-periods. The number of virtual analysis units required for the inspection items is positively correlated with the algorithm model complexity of the algorithm model included in the inspection item information of the inspection items, and the number of virtual analysis units required for the inspection items whose analysis mode is the real-time analysis mode is higher than the number of virtual analysis units required for the inspection items whose analysis mode is the screenshot analysis mode. The real-time analysis mode and the screenshot analysis mode will be explained in detail below and will not be repeated here.
[0096] S1013, determining the maximum value of the second numbers of all sub-time periods as the third number; and for each inspection item in the first inspection plan, determining the number of virtual analysis units required to be occupied by the inspection item, and taking the maximum value of the determined numbers of virtual analysis units as the fourth number;
[0097] S1014: If the sum of the third number and the fourth number is not greater than the total number of virtual analysis units in the first physical analysis unit, store the first inspection plan.
[0098] If the sum of the third number and the fourth number is greater than the total number of virtual analysis units in the first physical analysis unit, the first inspection plan is rejected for storage. In this case, resource conflicts may occur when executing the first inspection plan.
[0099] For example, see Figure 5 , Figure 5 A schematic diagram of a conflict check provided in an embodiment of the present application, Figure 5 Taking Monday as an example, the first execution period of the first inspection plan is 9:00-19:00. The first physical analysis unit to be applied to execute the first inspection plan is physical analysis unit 1. Physical analysis unit 1 includes 4 virtual analysis units, namely virtual analysis unit 1, virtual analysis unit 2, virtual analysis unit 3 and virtual analysis unit 4. The first inspection plan includes 2 inspection items, namely inspection item 1 and inspection item 2. The execution order of inspection item 1 is 1 and 3, the execution time is 3 hours, and the number of virtual analysis units occupied by inspection item 1 is 1. The execution order of inspection item 2 is 2, the execution time is 4 hours, and the number of virtual analysis units occupied by inspection item 2 is 2. Based on this, all sub-periods in the first execution period include sub-period 1 (9:00-12:00), sub-period 2 (12:00-16:00) and sub-period 3 (16:00-19:00).
[0100] The execution period of the second inspection plan is recorded as the second execution period, which is 16:00-24:00. The physical analysis unit to be applied to execute the second inspection plan is also physical analysis unit 1. The first inspection plan includes 2 inspection items, namely inspection item 4 and inspection item 5. The execution order of inspection item 4 is 1, the execution time is 3 hours, and the number of virtual analysis units required to execute inspection item 4 is 2. The execution order of inspection item 5 is 2, the execution time is 5 hours, and the number of virtual analysis units required to execute inspection item 5 is 2. Based on this, all sub-periods in the second execution sub-period include sub-period 4 (time is 16:00-19:00) and sub-period 5 (time is 19:00-24:00).
[0101] For sub-period 1 and sub-period 2 in the first execution period, there is no inspection item executed in the sub-period in the second inspection plan, and for sub-period 3, the inspection item executed in the sub-period in the second inspection plan is inspection item 4, then the second number of the sub-period is 2, and the inspection item executed in the sub-period is only inspection item 4, so the third number is 2. For each inspection item in the first inspection plan, the fourth number is determined to be 2, and the sum of the third data and the fourth number is equal to the total number of virtual analysis units in the first physical analysis unit, so there will be no resource conflict when executing the first inspection plan, and the inspection device can store the first inspection plan in the storage medium.
[0102] Applying the above embodiment, the method automatically retrieves and summarizes the inspection items and their resource requirements in the same time period in all the stored inspection plans related to the first image acquisition device, and by accurately calculating the maximum occupancy of the virtual analysis unit required for the inspection items in each sub-period, and comparing it with the newly input inspection plan resource requirements, the method can instantly determine whether the system resources are sufficient to support the execution of the new plan. If the resources are sufficient, the new plan will be stored smoothly; if the resources are insufficient, the storage will be refused, avoiding the failure or delay of the inspection task execution due to resource conflicts. This mechanism not only ensures the smooth execution of the inspection task, but also optimizes resource allocation, reduces the degradation of system performance due to insufficient resources, and thus improves the efficiency and reliability of the overall inspection work.
[0103] After verifying that there is no conflict between the first inspection plan and the stored inspection plan in the above manner, the inspection device can automatically specify a physical analysis unit for the first inspection plan and a virtual analysis unit for each inspection item in the plan, as follows:
[0104] For each inspection item in the first inspection plan, the number of virtual analysis units required to be occupied by the inspection item is determined, and the maximum value of the determined numbers of virtual analysis units is used as the first number.
[0105] A first number of virtual analysis units to which no inspection plan is allocated in a first execution period are determined in the first physical analysis unit as first virtual analysis units, and a first inspection plan is allocated to all first virtual analysis units.
[0106] After allocating each inspection item in the first inspection plan to a corresponding virtual analysis unit, the aforementioned step S103 includes:
[0107] S1031, controlling all first virtual analysis units in the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point;
[0108] S1032, analyzing the sampled image data through all first virtual analysis units in the first physical analysis unit after the algorithm model is loaded to obtain an inspection result.
[0109] For example, referring to the above example, the first execution period of the first inspection plan is 9:00-11:00, and the first physical analysis unit to be applied to execute the first inspection plan is physical analysis unit 1. Physical analysis unit 1 includes 4 virtual analysis units, namely virtual analysis unit 1, virtual analysis unit 2, virtual analysis unit 3 and virtual analysis unit 4. The first inspection plan includes 2 inspection items, namely inspection item 1 and inspection item 2. The execution order of inspection item 1 is 1 and 3, the execution time is 30 minutes, and the number of virtual analysis units required to execute inspection item 1 is 1. The execution order of inspection item 2 is 2, the execution time is 1 hour, and the number of virtual analysis units required to execute inspection item 2 is 2. Based on this, it is determined that the first number is 2, and the two virtual analysis units that are not assigned to the inspection plan of physical analysis unit 1 in the first execution period of 9:00-11:00 are determined as the first virtual analysis unit, and the first inspection plan is assigned to the first virtual analysis unit.
[0110] After completing the allocation of inspection plans according to the above method, the allocation of inspection plans can be expressed in terms of time dimension and analysis unit dimension, see Figure 6 , Figure 6 The first schematic diagram of the inspection plan allocation provided in the embodiment of the present application represents the allocation of the inspection plan in the time dimension. The physical analysis unit 1 executes the inspection plan 1, inspection plan 2, inspection plan 3 and inspection plan 4 respectively from Monday to Friday. The table items corresponding to each date are divided into four rows, and each row represents a virtual analysis unit. The execution period of the inspection plan 1 is the time period from Monday to Friday 00:00-07:00, and all the inspection items in the inspection plan 1 occupy 1 virtual analysis unit; the execution period of the inspection plan 2 is the time period from Monday to Friday 07:00-18:00, and all the inspection items in the inspection plan 2 occupy 2 virtual analysis units; the execution period of the inspection plan 3 is the time period from Monday to Friday 18:00-24:00, and all the inspection items in the inspection plan 3 occupy 2 virtual analysis units; the execution period of the inspection plan 4 is the time period from Monday to Friday 00:00-24:00, and all the inspection items in the inspection plan 4 occupy 2 virtual analysis units.
[0111] like Figure 6As shown, when the user enters the first inspection plan for the first image acquisition device through the configuration interface in the super brain web page or the configuration interface on the server side, the 24 hours of each day are divided into 24 time periods, including 00:00-01:00, 01:00-02:00, 02:00-03:00, ..., 23:00-24:00, and so on. Exemplarily, the 24 hours of each day can also be divided into any number of time periods. When entering the first inspection plan, the start and end times can be arbitrarily divided according to user needs. In order to facilitate user management, the start and end times are accurate to minutes, and the same inspection plan cannot be assigned to two days for execution, that is, a single execution period is not allowed to cross days.
[0112] See also Figure 7 , Figure 7 A second schematic diagram of inspection plan allocation provided in an embodiment of the present application, in which the allocation of inspection plans is represented in the dimension of physical analysis units. Physical analysis unit 1 and physical analysis unit 2 respectively execute inspection plan 1, inspection plan 2, inspection plan 3, inspection plan 4, inspection plan 5 and inspection plan 6 on Monday. The table items corresponding to each physical analysis unit are divided into four rows, each row representing a virtual analysis unit, wherein the execution period of inspection plan 1 is the time period 00:00-07:00 on Monday, and all inspection items in inspection plan 1 occupy one virtual analysis unit of physical analysis unit 1; the execution period of inspection plan 2 is the time period 07:00-18:00 on Monday, and all inspection items in inspection plan 2 occupy two virtual analysis units of physical analysis unit 1. analysis unit; the execution period of inspection plan 3 is 18:00-24:00 on Monday, and all inspection items in inspection plan 3 occupy 2 virtual analysis units of physical analysis unit 1; the execution period of inspection plan 4 is 00:00-24:00 on Monday, and all inspection items in inspection plan 4 occupy 2 virtual analysis units of physical analysis unit 1; the execution period of inspection plan 5 is 00:00-07:00 and 17:00-24:00 on Monday, and all inspection items in inspection plan 5 occupy 4 virtual analysis units of physical analysis unit 2; the execution period of inspection plan 6 is 07:00-17:00 on Monday, and all inspection items in inspection plan 6 occupy 4 virtual analysis units of physical analysis unit 2.
[0113] like Figure 7As shown, the allocation of the inspection plan is expressed in the time dimension in the same way. When the user enters the first inspection plan for the first image acquisition device through the configuration interface in the super brain web page or the configuration interface on the server side, the 24 hours of each day are still divided into 24 time periods, including 00:00-01:00, 01:00-02:00, 02:00-03:00, ..., 23:00-24:00. Exemplarily, the 24 hours of each day can also be divided into any number of time periods. When entering the first inspection plan, the start and end times can be arbitrarily divided according to user needs. In order to facilitate user management, the start and end times are accurate to minutes, and the same inspection plan cannot be assigned to two days for execution, that is, a single execution period is not allowed to cross days.
[0114] After allocating each inspection item in the first inspection plan to the first virtual analysis unit, the aforementioned step S103 includes:
[0115] Control all first virtual analysis units in the first physical analysis unit to load the read algorithm model, and control the first image acquisition device to switch to the read shooting point; analyze the sampled image data through all first virtual analysis units in the first physical analysis unit after loading the algorithm model to obtain the inspection result.
[0116] In S102, the inspection device periodically detects whether the first execution period has arrived. In a possible embodiment, the user can send a detection instruction to the inspection device through the server or the super brain terminal based on the start time of the first execution period before the start time is about to arrive, so that the inspection device detects whether the first execution period has arrived according to the preset cycle. However, adopting this method will increase the burden on users, affect user experience, and also increase inspection costs.
[0117] Based on this, in another possible embodiment, see Figure 8 , Figure 8 The third structural diagram of the inspection device provided in the embodiment of the present application, the inspection device also runs a thread 600 for each physical analysis unit 200, and the inspection device periodically wakes up the first thread running for the first physical analysis unit to obtain the system time of the inspection device through the first thread, and determines through the first thread whether the difference between the start time of the first execution period and the system time is within a preset difference range. If the difference is within the preset difference range, it is determined that the first execution period has arrived.
[0118] By applying the above embodiment, by allocating a separate thread to each physical analysis unit, the inspection device can process multiple inspection tasks in parallel, significantly improving the inspection efficiency and response speed. By periodically waking up the first thread for the first physical analysis unit to obtain the system time, and judging whether the first execution period has arrived, not only the punctuality of the inspection is improved, but also it helps to optimize the allocation of inspection resources, reduce unnecessary waiting and delays, improve the inspection effect, and reduce the inspection cost.
[0119] When the first execution period arrives, the inspection device reads the analysis mode, algorithm model and shooting points recorded in the inspection item information of the first inspection item in the inspection plan, and executes step S103.
[0120] In S103, the inspection device controls the first physical analysis unit to load the read algorithm model, and controls the first image acquisition device to switch to the read shooting point. After each inspection item in the first inspection plan is assigned to the corresponding virtual analysis unit through the aforementioned method, the inspection item can be executed through the aforementioned steps S1031 and S1032, and the inspection result can be obtained.
[0121] When executing each inspection item, corresponding to the different analysis modes of the inspection item, the inspection device selects different methods to sample the image data collected by the first image acquisition device. When the analysis mode of the inspection item is the real-time analysis mode, the inspection device will sample the image data collected by the first image acquisition device after the switching point in real time, that is, obtain all the image data collected by the first image acquisition device. When the analysis mode of the inspection item is the snapshot analysis mode, the inspection device will sample the image data collected by the first image acquisition device after the switching point according to the snapshot interval indicated by the snapshot analysis mode, that is, regularly obtain part of the image data collected by the first image acquisition device.
[0122] By applying the above embodiments, the inspection equipment can dynamically adjust the sampling strategy of image data according to actual needs by automatically identifying and applying different analysis modes (real-time analysis mode and snapshot analysis mode). In the real-time analysis mode, the inspection equipment can process image data in real time to ensure the immediacy and accuracy of the inspection, which is particularly suitable for scenarios that require rapid response. In the snapshot analysis mode, the inspection equipment can sample according to the preset snapshot interval, effectively reducing data redundancy, reducing storage and processing costs, and ensuring the capture of key information. It is suitable for periodic inspections or long-term monitoring tasks, which not only improves resource utilization, but also enhances the adaptability and scalability of inspections.
[0123] In S104, the inspection device also periodically obtains the duration of execution of the current inspection item by the first physical analysis unit. If the execution duration exceeds the planned execution duration of the inspection item, the inspection item is stopped and the inspection item with the next highest priority is executed.
[0124] In S105 , when the execution period of the inspection plan ends, the first physical analysis unit is controlled to terminate the execution of the inspection item currently being executed.
[0125] According to the above S104 and S105, the inspection method provided in the present application can stop executing the inspection item after the planned duration of the currently executed inspection item ends, or can stop executing the inspection item when the execution period of the inspection plan to which the currently executed inspection item belongs ends.
[0126] In another possible embodiment, the inspection device may detect in real time whether the first image acquisition device is located at the shooting point corresponding to the target inspection item during the process of the first physical analysis unit executing the target inspection item. If it is detected that the first image acquisition device has switched positions, that is, is not located at the shooting point corresponding to the target inspection item, the first physical analysis unit is controlled to suspend execution of the target inspection item.
[0127] When it is detected that the first image acquisition device switches the shooting point back to the shooting point corresponding to the target inspection item, the first physical analysis unit is controlled to continue executing the target inspection item.
[0128] By applying the above embodiment, when the first image acquisition device needs to switch points during the inspection process, the continuity and accuracy of the image data can be ensured to be undisturbed by timely pausing the execution of the target inspection item by the first physical analysis unit. This intelligent pause mechanism effectively avoids the confusion or loss of image data caused by point switching, thereby ensuring the accuracy of subsequent inspection results. At the same time, when the first image acquisition device switches back to the preset shooting point, the inspection device can automatically resume the execution of the target inspection item by the first physical analysis unit without manual intervention, thereby realizing the automation and seamless connection of the inspection process. This not only improves the inspection efficiency and accuracy, but also reduces the burden on users, making the entire inspection process smoother and more efficient, and improving the user experience.
[0129] After executing each inspection item and obtaining the inspection result, in one possible embodiment, the inspection device can directly display the inspection result or upload the inspection result to the server. In another possible embodiment, see Fig. 9 , Fig. 9The fourth structural diagram of the inspection device provided in the embodiment of the present application, the inspection device is further provided with a network unit 700, and the network unit 700 is used to establish a communication connection between the inspection device and the alarm center 800. Based on this structure, the inspection device can also display the alarm mode control in response to the enabling of the proprietary alarm mode, and in response to the configuration instruction for the alarm mode control, obtain and record the first alarm mode indicated by the configuration instruction; if the inspection result meets the preset alarm condition and the proprietary alarm mode is enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition and the proprietary alarm mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode.
[0130] For example, taking the inspection device inspecting machine A on a production line, suppose the inspection result shows that the temperature of the machine rises abnormally and exceeds the preset safety threshold, which means that the preset alarm condition is met.
[0131] After starting the dedicated alarm mode, the inspection device will send an alarm message to the alarm center through the communication connection in the first alarm mode, such as sending an encrypted emergency alarm email.
[0132] When the dedicated alarm mode is not activated, the inspection device will send an alarm message to the alarm center through a communication connection in accordance with the second alarm mode, such as sending a common SMS alarm.
[0133] The inspection device can also send different alarm information to the alarm center in different alarm modes according to the user's choice, whether the proprietary alarm mode is enabled, and whether the superposition detection frame is enabled. There are four specific situations:
[0134] Case 1: If the inspection result meets the preset alarm condition, and the proprietary alarm mode is enabled and the superimposed detection frame mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data on which the inspection result is based.
[0135] Case 2: If the inspection results meet the preset alarm conditions, and the proprietary alarm mode has been enabled and the superimposed detection frame mode has been enabled, an alarm message is sent to the alarm center through the communication connection in accordance with the first alarm method. The alarm message includes the image data on which the inspection results are based and the detection frame of the object that meets the alarm conditions.
[0136] Case three: If the inspection result meets the preset alarm conditions, and the proprietary alarm mode is not enabled and the superimposed detection frame mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode. The alarm message includes the image data on which the inspection result is based.
[0137] Case 4: If the inspection result meets the preset alarm conditions, the proprietary alarm mode is not enabled and the superimposed detection frame mode is enabled, an alarm message is sent to the alarm center through a communication connection in accordance with the preset second alarm mode. The alarm message includes the image data on which the inspection result is based and the detection frame of the object that meets the alarm conditions.
[0138] For example, still taking the example that the inspection equipment is inspecting machine A on the production line, suppose the inspection result shows that the temperature of the machine rises abnormally and exceeds the preset safety threshold, which means that the preset alarm condition is met.
[0139] In situation one, the inspection device will send an alarm message to the alarm center through a communication connection in accordance with the first alarm method, such as sending an encrypted emergency alarm email, and the alarm message includes high-definition image data for machine A.
[0140] In situation two, the inspection equipment will send an alarm message to the alarm center through a communication connection in accordance with the first alarm method, such as sending an encrypted emergency alarm email. The alarm message includes high-definition image data for machine A and a target detection box will be used to select machine A in the image data.
[0141] In situation three, the inspection equipment will use the second alarm method, such as sending a normal SMS alarm, to send an alarm message to the alarm center through a communication connection, and the alarm message includes high-definition image data for machine A.
[0142] In situation four, the inspection equipment will use the second alarm method, such as sending a normal SMS alarm, to send an alarm message to the alarm center through a communication connection. The alarm message includes high-definition image data for machine A and a target detection box will be used to select machine A in the image data.
[0143] By applying the above-mentioned embodiments, the inspection efficiency and problem response speed are significantly improved through the integrated network unit and the flexibly configured alarm mode. When the proprietary alarm mode is enabled, the user can customize an efficient first alarm mode to ensure that the alarm center is notified quickly in an emergency, and attach detailed image data and abnormal object detection boxes to facilitate accurate positioning and rapid processing. In non-proprietary mode, a preset second alarm mode is also provided to ensure basic communication needs. The superimposed detection box mode further enhances the intuitiveness of the alarm information and helps the recipient quickly identify the problem area. It not only improves the level of inspection automation, but also effectively shortens the time from problem discovery to resolution through the intelligent alarm mechanism, providing a strong guarantee for industrial safety and production efficiency.
[0144] In order to more clearly illustrate the inspection method of the embodiment of the present application, the inspection method will be exemplarily illustrated below in combination with the inspection equipment mentioned above.
[0145] See also Fig.10 , Fig.10 The second flow chart of the inspection method provided in the embodiment of the present application specifically includes the following steps:
[0146] S1001, obtaining an inspection task plan from a storage medium, that is, obtaining and storing a first inspection plan input by a user for a first image acquisition device;
[0147] S1002, when the planned time point is reached, the inspection task information is obtained, that is, the aforementioned periodic detection is performed to determine whether the first execution period has arrived. If it has arrived, the analysis mode, algorithm model and shooting points recorded in the inspection item information of the target inspection item are read;
[0148] S1003, switching the preset point of the ball camera according to the shooting point configured for the inspection task, that is, controlling the first image acquisition device to switch to the read shooting point;
[0149] S1004, obtaining an idle virtual analysis unit and loading an algorithm model, that is, controlling all first virtual analysis units in the first physical analysis unit to load the read algorithm model;
[0150] S1005, starting different scene analysis tasks, that is, all the first virtual analysis units in the first physical analysis unit after loading the algorithm model analyze the sampled image data to obtain the inspection result.
[0151] Corresponding to the aforementioned inspection method, the present application also provides an inspection device, see Fig.11 , Fig.11 A schematic diagram of the structure of the inspection device provided in the embodiment of the present application is applied to an inspection device including multiple device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having multiple shooting points, and the device includes:
[0152] The acquisition module 1101 is used to acquire and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied for executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: execution sequence, execution duration, analysis mode and algorithm model to be applied for executing the inspection item, and a shooting point position at which the first image acquisition device needs to be when executing the inspection item;
[0153] The detection module 1102 is used to periodically detect whether the first execution period has arrived. If it has arrived, the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item are read. Initially, the target inspection item is the inspection item with the first execution order;
[0154] The execution module 1103 is used to control the first physical analysis unit to load the read algorithm model, and control the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data by the first physical analysis unit after the algorithm model is loaded to obtain the inspection result;
[0155] The patrol module 1104 is used to periodically obtain the duration of the target inspection item that the first physical analysis unit has executed. If the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, the first physical analysis unit is controlled to terminate the execution of the target inspection item, and the inspection item whose execution order is next to the target inspection item is used as the new target inspection item, and the step of returning to execute the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item is read;
[0156] The termination module 1105 is configured to control the first physical analysis unit to terminate execution of the target inspection item until the first execution period ends.
[0157] By using the inspection device provided in the embodiment of the present application, by presetting a detailed inspection plan, including an execution period, a physical analysis unit, inspection items and their inspection item information, the flexibility and pertinence of the inspection are greatly improved, and the comprehensiveness and efficiency of the inspection work are ensured. Secondly, through the periodic detection and execution mechanism, it is ensured that the inspection task can be automatically started on time, and is strictly executed in accordance with the preset analysis mode, algorithm model and shooting point, which effectively reduces human intervention and improves the accuracy of the inspection. Furthermore, by real-time control of the shooting point of the image acquisition device and the loading of the algorithm model of the physical analysis unit, the dynamic adjustment and optimization of the inspection process is realized, and the efficiency of the inspection is further improved. Finally, it can automatically terminate and switch to the next inspection item according to the execution time of the inspection item, effectively avoiding unnecessary waste of resources and redundancy of the inspection process, avoiding ineffective occupation of resources, and ensuring the stable operation of the inspection equipment. To sum up, the inspection device realizes flexible scheduling and inspection analysis in multiple point scenarios within a time period using multiple algorithm models. This automated and intelligent inspection method reduces manual intervention, reduces labor costs, avoids ineffective occupation of resources, improves the efficiency and stability of inspection work, and provides more efficient inspection solutions for various application scenarios.
[0158] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above inspection methods are implemented.
[0159] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any inspection method in the above embodiments.
[0160] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.
[0161] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0162] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0163] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims
1. A patrol inspection method, characterized in that: Applicable to an inspection device including a plurality of device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having a plurality of shooting points; The method comprises: Acquire and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied when executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: execution order, execution duration, analysis mode and algorithm model to be applied when executing the inspection item, and a shooting point position at which the first image acquisition device needs to be when executing the inspection item; Periodically detecting whether the first execution period has arrived, and if it has arrived, reading the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item, and initially the target inspection item is the inspection item with the first execution order; Control the first physical analysis unit to load the read algorithm model, and control the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data through the first physical analysis unit after the algorithm model is loaded to obtain the inspection result; Periodically obtain the duration of the target inspection item that the first physical analysis unit has executed. If the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, control the first physical analysis unit to terminate the execution of the target inspection item, and use the inspection item whose execution order is next to the target inspection item as the new target inspection item, and return to the step of executing the analysis mode, algorithm model and shooting point recorded in the inspection item information of the target inspection item; until the first execution period has ended, controlling the first physical analysis unit to terminate execution of the target inspection item; At least one virtual analysis unit is running in each of the physical analysis units, and the method further includes: The obtaining and storing a first inspection plan input by a user for the first image acquisition device includes: Acquire a first inspection plan input by a user for the first image acquisition device, and search for all second inspection plans including the first physical analysis unit in the stored inspection plans; For all sub-periods in the first execution period, respectively, search for all inspection items executed in the sub-periods in all the second inspection plans, and determine the sum of the numbers of virtual analysis units required to be occupied by all the found inspection items as the second number of the sub-periods; Determine the maximum value of the second number of all sub-periods as the third number; and for each inspection item in the first inspection plan, determine the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the fourth number; If the sum of the third number and the fourth number is not greater than the total number of virtual analysis units in the first physical analysis unit, storing the first inspection plan; The method further comprises: If the sum of the third number and the fourth number is greater than the total number of virtual analysis units in the first physical analysis unit, refusing to store the first inspection plan; Among them, the number of virtual analysis units required for the inspection item is positively correlated with the algorithm model complexity of the algorithm model included in the inspection item information of the inspection item, and the number of virtual analysis units required for the inspection item whose analysis mode is the real-time analysis mode is higher than the number of virtual analysis units required for the inspection item whose analysis mode is the screenshot analysis mode.
2. The inspection method according to claim 1, characterized in that: The method further comprises: If the first image acquisition device is detected to switch a point during the process of the first physical analysis unit executing the target inspection item, the first physical analysis unit is controlled to suspend execution of the target inspection item; If the first image acquisition device is switched back to the read shooting point, the first physical analysis unit is controlled to continue to execute the target inspection item.
3. The inspection method according to claim 1, characterized in that: The method further comprises: For each inspection item in the first inspection plan, determine the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the first number; Determine, in the first physical analysis unit, the first number of virtual analysis units to which no inspection plan is allocated during the first execution period as first virtual analysis units, and allocate the first inspection plan to all the first virtual analysis units; The controlling the first physical analysis unit to load the read algorithm model and controlling the first image acquisition device to switch to the read shooting point position includes: Controlling all the first virtual analysis units in the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point; The first physical analysis unit after loading the algorithm model analyzes the sampled image data to obtain an inspection result, including: All the first virtual analysis units in the first physical analysis unit after loading the algorithm model analyze the sampled image data to obtain an inspection result.
4. The inspection method according to claim 1, characterized in that: The inspection device runs a thread for each physical analysis unit; the method further includes: Periodically waking up a first thread running on the first physical analysis unit to obtain the system time of the inspection device through the first thread; and determining, through the first thread, whether a difference between the start time of the first execution period and the system time is within a preset difference range, and if so, determining that the first execution period has arrived; The step of sampling the image data acquired by the first image acquisition device after the switching point according to the read analysis mode includes: If the analysis mode read is the real-time analysis mode, the image data collected by the first image acquisition device after the switching point is sampled in real time; if the analysis mode read is the snapshot analysis mode, the image data collected by the first image acquisition device after the switching point is sampled periodically according to the snapshot interval indicated by the snapshot analysis mode.
5. The inspection method according to claim 1, characterized in that: The inspection device further includes a network unit, and the network unit is used to establish a communication connection between the inspection device and the alarm center; the method further includes: In response to the enabling of the proprietary alarm mode, an alarm mode control is displayed; in response to a configuration instruction for the alarm mode control, a first alarm mode indicated by the configuration instruction is acquired and recorded; if the inspection result meets the preset alarm condition and the proprietary alarm mode is enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition and the proprietary alarm mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode; In response to the enabling of the superimposed detection frame mode, if the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is not enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is enabled, an alarm message is sent to the alarm center through the communication connection according to the first alarm mode, and the alarm message includes the image data based on which the inspection result is obtained and the corresponding image data that meets the alarm condition. an object detection frame; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode, and the alarm message includes the image data on which the inspection result is based; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode, and the alarm message includes the image data on which the inspection result is based and the detection frame of the object that meets the alarm condition.
6. A patrol inspection device, characterized in that: The inspection device includes a plurality of device interfaces and at least one physical analysis unit, wherein the device interface is used to access an image acquisition device, wherein the image acquisition device includes a first image acquisition device having a plurality of shooting points; The inspection device is used to obtain and store a first inspection plan input by a user for the first image acquisition device, wherein the first inspection plan includes: a first execution period of the first inspection plan, a first physical analysis unit to be applied for executing the first inspection plan, at least one inspection item, and inspection item information of each inspection item, wherein the inspection item information includes: execution order, execution duration, analysis mode and algorithm model to be applied for executing the inspection item, and shooting point position at which the first image acquisition device needs to be when executing the inspection item; periodically detecting whether the first execution period has arrived, and if so, reading the analysis mode, algorithm model and shooting point position recorded in the inspection item information of the target inspection item, wherein the target inspection item is the inspection item with the first execution order; controlling the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point; and execute the target inspection item in the following manner: sample the image data collected by the first image acquisition device after the switching point according to the read analysis mode, and analyze the sampled image data by the first physical analysis unit after loading the algorithm model to obtain the inspection result; periodically obtain the duration of the first physical analysis unit executing the target inspection item, if the obtained duration is not less than the execution duration recorded in the inspection item information of the target inspection item, control the first physical analysis unit to terminate the execution of the target inspection item, and use the inspection item with the execution order next to the target inspection item as the new target inspection item, and return to the step of executing the analysis mode, algorithm model and shooting point recorded in the inspection item information of the read target inspection item; until the first execution period has ended, control the first physical analysis unit to terminate the execution of the target inspection item; The first image acquisition device is used to respond to the control of the inspection device, switch to a desired shooting point, and acquire image data at the shooting point; At least one virtual analysis unit is running in each of the physical analysis units, and the inspection device is further used for: The obtaining and storing a first inspection plan input by a user for the first image acquisition device includes: Acquire a first inspection plan input by a user for the first image acquisition device, and search for all second inspection plans including the first physical analysis unit in the stored inspection plans; For all sub-periods in the first execution period, respectively, search for all inspection items executed in the sub-periods in all the second inspection plans, and determine the sum of the numbers of virtual analysis units required to be occupied by all the found inspection items as the second number of the sub-periods; Determine the maximum value of the second number of all sub-periods as the third number; and for each inspection item in the first inspection plan, determine the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the fourth number; If the sum of the third number and the fourth number is not greater than the total number of virtual analysis units in the first physical analysis unit, storing the first inspection plan; The inspection device is further configured to refuse to store the first inspection plan if the sum of the third number and the fourth number is greater than the total number of virtual analysis units in the first physical analysis unit; Among them, the number of virtual analysis units required for the inspection item is positively correlated with the algorithm model complexity of the algorithm model included in the inspection item information of the inspection item, and the number of virtual analysis units required for the inspection item whose analysis mode is the real-time analysis mode is higher than the number of virtual analysis units required for the inspection item whose analysis mode is the screenshot analysis mode.
7. The inspection device according to claim 6, characterized in that: The inspection device is further used to determine, for each inspection item in the first inspection plan, the number of virtual analysis units required to be occupied by the inspection item, and use the maximum value of the determined numbers of virtual analysis units as the first number; Determine, in the first physical analysis unit, the first number of virtual analysis units to which no inspection plan is allocated during the first execution period as first virtual analysis units, and allocate the first inspection plan to all the first virtual analysis units; The controlling the first physical analysis unit to load the read algorithm model and controlling the first image acquisition device to switch to the read shooting point position includes: Controlling all the first virtual analysis units in the first physical analysis unit to load the read algorithm model, and controlling the first image acquisition device to switch to the read shooting point; The first physical analysis unit after loading the algorithm model analyzes the sampled image data to obtain an inspection result, including: All the first virtual analysis units in the first physical analysis unit after loading the algorithm model analyze the sampled image data to obtain an inspection result.
8. The inspection device according to claim 6, characterized in that: The inspection device runs a thread for each physical analysis unit; The inspection device is further used to periodically wake up the first thread running for the first physical analysis unit to obtain the system time of the inspection device through the first thread; and determine through the first thread whether the difference between the start time of the first execution period and the system time is within a preset difference range, and if so, determine that the first execution period has arrived; The step of sampling the image data acquired by the first image acquisition device after the switching point according to the read analysis mode includes: If the read analysis mode is the real-time analysis mode, sampling the image data collected by the first image acquisition device after the switching point in real time; If the read analysis mode is a snapshot analysis mode, the image data collected by the first image acquisition device after the switching point is sampled regularly according to the snapshot interval indicated by the snapshot analysis mode.
9. The inspection device according to claim 6, characterized in that: The inspection device further comprises a network unit, which is used to establish a communication connection between the inspection device and an alarm center; The inspection device is further used to display an alarm mode control in response to the enabling of a proprietary alarm mode; in response to a configuration instruction for the alarm mode control, obtain and record a first alarm mode indicated by the configuration instruction; if the inspection result meets a preset alarm condition and the proprietary alarm mode is enabled, send an alarm message to the alarm center through the communication connection in accordance with the first alarm mode, wherein the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition and the proprietary alarm mode is not enabled, send an alarm message to the alarm center through the communication connection in accordance with a preset second alarm mode; The inspection device is also used to respond to the enabling of the superimposed detection frame mode. If the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is not enabled, then an alarm message is sent to the alarm center through the communication connection according to the first alarm method, and the alarm message includes the image data based on which the inspection result is obtained; if the inspection result meets the preset alarm condition, and the dedicated alarm mode is enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the first alarm method, and the alarm message includes the image data based on which the inspection result is obtained and the image data that meets the alarm condition. a detection frame of an object meeting the alarm condition; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to a preset second alarm mode, and the alarm message includes the image data on which the inspection result is based; if the inspection result meets the preset alarm condition, and the exclusive alarm mode is not enabled and the superimposed detection frame mode is enabled, then an alarm message is sent to the alarm center through the communication connection according to the preset second alarm mode, and the alarm message includes the image data on which the inspection result is based and the detection frame of the object meeting the alarm condition.
