Method, control device, system and storage medium for monitoring an industrial device inspection

Abstract

The embodiment of the present application provides a kind of method, control device, system and storage medium for monitoring the inspection of industrial equipment, belong to industrial equipment detection technical field.The method comprises: for the video data of inspection task, multiple video frames are collected;Feature points of each video frame are extracted;By comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database, the industrial equipment matched by each video frame is determined;And when the industrial equipment matched by multiple video frames contains all industrial equipment in the inspection task, it is determined that the inspection task is completed.The embodiment of the present application is based on feature point matching algorithm, by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the industrial equipment feature database, the industrial equipment matched by each video frame is quickly determined, to monitor the inspection task.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial equipment detection, and in particular to a method, a control device, a system and a storage medium for monitoring industrial equipment inspection. Background Art

[0002] In industrial environments, maintenance personnel are often required to conduct regular inspections of industrial equipment. To strengthen and standardize the management of maintenance personnel's inspections, it is necessary to equip maintenance personnel with professional cameras to determine whether they are conducting rigorous inspections of industrial equipment.

[0003] Currently, deep learning-based object detection algorithms are often used to detect captured industrial equipment images to determine whether maintenance personnel have inspected all industrial equipment. However, due to the large differences in industrial equipment and complex industrial environments, existing object detection algorithms require large datasets to be prepared in advance for training, resulting in high costs. Furthermore, changes to industrial equipment require retraining of the object detection model, slowing down the iteration speed of the detection system. Summary of the Invention

[0004] An object of the embodiments of the present invention is to provide a method for monitoring industrial equipment inspections, which can quickly monitor inspection tasks.

[0005] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a method for monitoring industrial equipment inspections, which includes: collecting multiple video frames for video data corresponding to a preset inspection task; extracting feature points of each of the multiple video frames; determining the industrial equipment matched by each video frame by comparing the feature points of each video frame with a standard feature point set of each industrial equipment in a preset industrial equipment feature database; and when the industrial equipment matched by the multiple video frames includes all the industrial equipment in the preset inspection task, determining that the inspection task is completed.

[0006] Optionally, extracting feature points of each video frame from the multiple video frames includes: generating a corresponding image pyramid for each video frame, and extracting a first feature point set of the video frame in the image pyramid; labeling each pixel of the video frame with a background label or a non-background label through a background filtering algorithm; mapping pixels labeled with background labels to the first feature point set to determine the mapped feature points; and removing the mapped feature points from the first feature point set to determine the remaining feature points as the feature points extracted from the video frame.

[0007] Optionally, before the collecting of the plurality of video frames corresponding to the preset inspection task, the method for monitoring the inspection of the industrial equipment further comprises: creating the industrial equipment feature database; obtaining images of all industrial equipment in a preset range based on a plurality of angles; for each industrial equipment, constructing a corresponding image set according to the images taken from a plurality of angles; constructing a corresponding standard feature point set according to the image set of the industrial equipment; and storing the constructed standard feature point set and the corresponding industrial equipment in the industrial equipment feature database.

[0008] Optionally, the constructing of the corresponding image set for each industrial equipment according to the images taken from a plurality of angles comprises: extracting a corresponding second feature point set for each image in the image set; labeling each pixel of the image with a background label or a non-background label through a background filtering algorithm; mapping the pixels labeled as the background label to the second feature point set to determine the mapped feature points; removing the mapped feature points from the second feature point set to determine the remaining feature points as the extracted feature points of the image; and constructing the standard feature point set corresponding to the industrial equipment according to the extracted feature points of each image.

[0009] Optionally, the determining of the industrial equipment matched by each video frame by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database comprises: obtaining the standard feature point set corresponding to the industrial equipment to be inspected from the industrial equipment feature database according to the preset inspection task to form an industrial equipment feature data table; and comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the industrial equipment feature data table to determine the industrial equipment matched by each video frame.

[0010] Optionally, the determining of the industrial equipment matched by each video frame by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database comprises: calculating a first descriptor of the feature points of each video frame; and traversing the standard feature point set corresponding to each industrial equipment in the preset industrial equipment feature database, calculating a second descriptor of each standard feature point in each standard feature point set, counting the number of standard feature points with a distance between the first descriptor and the second descriptor less than a first preset threshold, and determining the standard feature point set with the largest number of counted standard feature points and exceeding a second preset threshold as the industrial equipment matched by the video frame.

[0011] Optionally, when the number of feature points of each video frame exceeds the preset number, the matching industrial equipment of each video frame is determined by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database, including: for each video frame, calculating a third descriptor of each feature point of the video frame; and traversing the standard feature point set corresponding to each industrial equipment in the preset industrial equipment feature database, for each sub-standard feature point set of the standard feature point set, calculating a second descriptor of each standard feature point in the sub-standard feature point set, calculating the distance between the third descriptor of each feature point and the second descriptor of each standard feature point in the sub-standard feature point set, and when there is a standard feature point with a distance less than the first preset threshold for each feature point, determining the industrial equipment corresponding to the sub-standard feature point set as the matching industrial equipment of the video frame.

[0012] Optionally, the video data corresponding to the preset inspection task is collected to obtain a plurality of video frames, including: obtaining the total duration T of the video data; determining the number N of industrial equipment to be inspected by the inspection task and the unique identifier of each industrial equipment; taking the first frame of the video data as a first sampling point, and sequentially determining other sampling points according to a preset step length, the preset step length = T / (N-1); and collecting the corresponding video frame based on each sampling point.

[0013] Optionally, the determination of the matching industrial equipment of each video frame further includes: for each sampling point, when the corresponding video frame matches an industrial equipment, recording the sampling point and the unique identifier of the corresponding industrial equipment; and when the corresponding video frame does not match an industrial equipment, recording the sampling point as unidentified.

[0014] Optionally, when the matching industrial equipment of the plurality of video frames does not contain all industrial equipment in the preset inspection task, the method for monitoring the inspection of industrial equipment further includes: according to the recorded sampling points, finding two adjacent sampling points that meet the following requirements: being marked with different unique identifiers of industrial equipment or being marked as unidentified; and the time interval between the two sampling points exceeding a preset time. Selecting two adjacent sampling points that meet the requirements and have the largest time interval; taking a point between the selected two adjacent sampling points as a new sampling point, collecting a new video frame based on the new sampling point; comparing the feature points of the new video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database to determine the matching industrial equipment of the new video frame; when the new video frame matches an industrial equipment, recording the new sampling point and the unique identifier of the corresponding industrial equipment; when the new video frame does not match an industrial equipment, recording the new sampling point as unidentified; and re-executing the above steps.

[0015] Optionally, when two adjacent sampling points that meet the requirements are not found, it is determined that the inspection task has failed.

[0016] An embodiment of the present invention also provides a control device for monitoring industrial equipment inspections, the control device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the above-mentioned method for monitoring industrial equipment inspections.

[0017] An embodiment of the present invention further provides a machine-readable storage medium, characterized in that the machine-readable storage medium stores instructions, which enable a machine to execute the above-mentioned method for monitoring industrial equipment inspections.

[0018] An embodiment of the present invention further provides a system for monitoring the inspection of industrial equipment. The system includes a camera device and the above-mentioned control device for monitoring the inspection of industrial equipment.

[0019] Through the above technical solution, the present embodiment collects multiple video frames based on the video data corresponding to the inspection task. Using a feature point matching algorithm, the feature points of each video frame are compared with the standard feature point set for each industrial device in a preset industrial device feature database to quickly determine the industrial device that matches each video frame for monitoring the inspection task. This eliminates the need to pre-generate large datasets. When industrial equipment changes, such as adding or removing industrial equipment, only the standard feature point set in the industrial device feature database needs to be added or removed, resulting in rapid iteration.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0022] Figure 1 This is a flow chart of a method for monitoring industrial equipment inspections provided by an embodiment of the present invention;

[0023] Figure 2 is a schematic structural diagram of a device provided by an embodiment of the present invention; and

[0024] Figure 3 It is a schematic diagram of the system structure for monitoring industrial equipment inspection provided by an embodiment of the present invention.

[0025] Description of Reference Numerals

[0026] 20 device 201 processor

[0027] 202 memory 203 bus

[0028] 31 Camera device 30 Control device for monitoring industrial equipment inspection DETAILED DESCRIPTION

[0029] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.

[0030] Figure 1 This is a flow chart of a method for monitoring industrial equipment inspection provided by an embodiment of the present invention. Please refer to Figure 1 , the method for monitoring industrial equipment inspection may include the following steps:

[0031] Step S110: For the video data corresponding to the preset inspection task, a plurality of video frames are collected.

[0032] To illustrate with an example, the maintenance personnel who perform the inspection task can use the camera device equipped with them to collect video data from the maintenance personnel's perspective. The video data can be obtained by shooting from the beginning of the inspection task to the end of the task; for the video data, video frame sampling can be performed according to a preset frequency (for example, it can be set according to the time that may be required to inspect each industrial equipment). The video data can also be a video segment shot from the beginning of the inspection task to the end of the task. For example, the video segment can be shot and saved according to the preset frequency; the video segment can also be shot and saved based on the acceleration change perceived by the acceleration sensor configured by the camera device. The change in acceleration may indicate that the maintenance personnel has replaced the industrial equipment being inspected. One or more video frames can also be collected for each video segment.

[0033] Step S120: extracting feature points of each video frame in the plurality of video frames.

[0034] Preferably, step S120 may include: generating a corresponding image pyramid for each video frame, extracting a first feature point set of the video frame in the image pyramid; labeling each pixel of the video frame with a background label or a non-background label through a background filtering algorithm; mapping the pixels labeled with the background label to the first feature point set to determine the mapped feature points; and removing the mapped feature points from the first feature point set to determine the remaining feature points as the feature points extracted from the video frame.

[0035] For example, for video frame Vf1, video frame Vf1 can be input into an image processing model to generate a corresponding image pyramid. The first feature point set of the original video frame in the image pyramid is extracted. Feature points are representative pixels selected from the original video frame in the corresponding image pyramid that reflect certain characteristics. Background filtering is performed to label each pixel in video frame Vf1 as either background or non-background. Background filtering can be performed using semantic segmentation algorithms and object detection algorithms. For example, semantic segmentation algorithms are used to classify each pixel in video frame Vf1 into two categories: pixels with background labels and pixels with non-background labels. All pixels in video frame Vf1 are mapped to the first feature point set, and feature points corresponding to pixels labeled as background are removed. That is, feature points at the same locations as pixels labeled as background are removed. The remaining feature points are determined as the feature points extracted for video frame Vf1. The number of feature points extracted for each video frame can be one or more.

[0036] Preferably, before step S110, the method for monitoring industrial equipment inspections may further include: creating the industrial equipment feature database; obtaining images of all industrial equipment within a preset range taken based on multiple angles; for each industrial equipment, constructing a corresponding image set based on the images taken from multiple angles; constructing the corresponding standard feature point set based on the image set of the industrial equipment; and storing the constructed standard feature point set and the corresponding industrial equipment in the industrial equipment feature database.

[0037] Further preferably, for each industrial equipment, a corresponding image set is constructed based on images taken from multiple angles thereof, including: for each image in the image set, extracting a corresponding second feature point set; labeling each pixel of the image with a background label or a non-background label through a background filtering algorithm; mapping the pixels labeled as background labels to the second feature point set to determine the mapped feature points; removing the mapped feature points from the second feature point set, and determining the remaining feature points as the feature points extracted from the image; and constructing the standard feature point set corresponding to the industrial equipment based on the feature points extracted from each image.

[0038] For example, images taken from multiple angles of all industrial equipment within a preset range are obtained. Similar to step S121, the corresponding standard feature point set is constructed based on the image set corresponding to each industrial equipment. For example, images I1-I1 taken from multiple angles of industrial equipment E1 are obtained. nFor the image I1, the image I1 can be input into the picture processing model, a corresponding image pyramid is generated, and a second feature point set of the original image in the image pyramid is extracted; each pixel point in the image I1 is labeled as a background label or a non-background label through background filtering; all pixels of the image I1 are mapped to the second feature point set, the feature points corresponding to the pixels labeled as the background label are removed, and the remaining feature points are determined as the feature points extracted from the image I1, and the feature points extracted from the image I1 are used as the feature points of the image I1-I n The standard feature point set of the industrial equipment E1 is constructed, and the standard feature point set is stored in the industrial equipment feature database in correspondence with the industrial equipment E1. The feature points extracted from each image can be one or more.

[0039] Preferably, the industrial equipment E1, the corresponding images I1-I n , the corresponding standard feature points, and the standard feature point set are stored in the industrial equipment feature database. The data format is, for example, industrial equipment E1, (image I1: standard feature point C1), (image I2: standard feature point C2), …, (image I n : standard feature point C n ), and standard feature point set (C1, C2, …, C n ). Each of C1, C2, …, C n may be one or more.

[0040] When an industrial equipment needs to be added, only the standard feature point set of the industrial equipment needs to be added in the industrial equipment feature database.

[0041] Step S130: Comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database, the industrial equipment matched by each video frame is determined.

[0042] Preferably, step S130 can include: for each video frame, calculating a first descriptor of the feature points of the video frame. The standard feature point set corresponding to each industrial equipment in the preset industrial equipment feature database is traversed, for each standard feature point set, a second descriptor of each standard feature point in the standard feature point set is calculated, the number of standard feature points with a distance between the first descriptor and the second descriptor less than a first preset threshold is counted, the standard feature point set with the largest number of counted standard feature points and exceeding a second preset threshold is determined, and the industrial equipment corresponding to the standard feature point set is determined as the industrial equipment matched by the video frame.

[0043] Preferably, the embodiment of the application matches the feature points, for each video frame, by comparing with the standard feature point set of each industrial equipment stored in the industrial equipment feature database, to obtain the industrial equipment successfully matched by the video frame.

[0044] For example, for the video frame Vf1, a first descriptor (corresponding to one or more) of the feature point (may be one or more) of the video frame Vf1 is calculated. The descriptor of the feature point can be understood as a mathematical representation unique to the corresponding feature point, usually in the form of a vector. Each standard feature point set corresponding to an industrial equipment in the industrial equipment feature database is traversed. For each standard feature point set, a second descriptor of each standard feature point is calculated. The distance (for example, the Euclidean distance) between each first descriptor and each second descriptor is calculated. The number of standard feature points with a distance between the descriptors less than a first preset threshold is counted. The standard feature point set with the largest number of counted standard feature points and exceeding a second preset threshold is determined. The industrial equipment corresponding to the standard feature point set is determined as the industrial equipment matched by the video frame. If the number of counted standard feature points of a certain standard feature point set is the largest but does not exceed the second preset threshold, the industrial equipment shown by the video frame may not be the industrial equipment under maintenance. Therefore, the video frame Vf1 is determined as not matched to an industrial equipment.

[0045] Preferably, when the number of feature points of each video frame exceeds a preset number, step S130 can include: for each video frame, calculating a third descriptor of each feature point of the video frame. Each standard feature point set corresponding to an industrial equipment in the preset industrial equipment feature database is traversed. For each sub-standard feature point set of each standard feature point set, a second descriptor of each standard feature point in the sub-standard feature point set is calculated. The distance between the third descriptor of each feature point and the second descriptor of each standard feature point in the sub-standard feature point set is calculated. When each feature point has a standard feature point with a distance less than the first preset threshold, the industrial equipment corresponding to the sub-standard feature point set is determined as the industrial equipment matched by the video frame.

[0046] As described above, for the industrial equipment E1, the images I1-I n , and the corresponding standard feature points can be collectively stored in the industrial equipment feature database. The data format is, for example, industrial equipment E1, (image I1: sub-standard feature point set C1), (image I2: sub-standard feature point set C2), …, (image I n : sub-standard feature point set C n ).

[0047] For example, when the video frame Vf1 includes K (K is an empirical value greater than 3, for example) feature points, when the K feature points are successfully matched with the sub-standard feature point set C n corresponding to the image (for example, image I n ) of a certain specific angle of a certain industrial equipment from the industrial equipment, the video frame Vf1 can be directly determined as successfully matched with the industrial equipment. For example, the sub-standard feature point set Cn There are B standard feature points, B ≥ K, if the K feature points of the video frame Vf1 are consistent with the sub-standard feature point set C n If K standard feature points out of B standard feature points are matched one by one (i.e., the distance between the corresponding descriptors is less than the first preset threshold), it can be directly determined that the video frame Vf1 is successfully matched with the industrial equipment E1, and the current traversal program can be terminated, thereby saving time consumption in the feature point matching process.

[0048] Preferably, step S130 may also include: according to the preset inspection task, obtaining the standard feature point set corresponding to the industrial equipment that needs to be inspected from the industrial equipment feature database to form an industrial equipment feature data table; and determining the industrial equipment matched by each video frame by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the industrial equipment feature data table.

[0049] To illustrate by example, the inspection task may only require inspecting some of the industrial equipment in the industrial equipment feature database. Therefore, the standard feature point set corresponding to the industrial equipment that needs to be inspected can be obtained according to the inspection task to form an industrial equipment feature data table. The feature points of each video frame are compared with the standard feature point set of each industrial equipment in the industrial equipment feature data table. The comparison process is as described above to determine the industrial equipment matched by each video frame, so as to save time consumption in the feature point matching process and improve matching efficiency.

[0050] Step S140: When the industrial equipment matched by the multiple video frames includes all the industrial equipment in the preset inspection task, it is determined that the inspection task is completed.

[0051] Continuing with the above example, based on multiple video frames collected by the inspection task, when the matched industrial equipment includes all industrial equipment in the industrial equipment feature data table, it is determined that the inspection task is completed, and a report is made that the inspection task has been completed.

[0052] Accordingly, the embodiment of the present invention collects multiple video frames based on the video data corresponding to the inspection task, and based on the feature point matching algorithm, compares the feature points of each video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database to quickly determine the industrial equipment matched by each video frame to monitor the inspection task. There is no need to prepare a large number of data sets in advance. When industrial equipment changes, such as adding or deleting industrial equipment, only the standard feature point set of the industrial equipment feature database needs to be added or deleted, and the iteration speed is fast. Furthermore, the embodiment of the present invention adopts a background filtering algorithm for each video frame, labels each pixel point in the video frame as a background label or a non-background label, maps pixels judged to be background labels to the feature point set, removes feature points with background labels, and removes complex industrial background environments to prevent background interference.

[0053] This embodiment of the present invention determines whether the video data corresponding to a preset inspection task contains all the industrial equipment to be inspected. As described in steps S110-S140, feature point matching is performed across multiple captured video frames to identify matching industrial equipment. Therefore, selecting the appropriate video frames from the video data significantly impacts the efficiency of the inspection task.

[0054] For example, once a video frame has been detected to match device A, it is not desired that the subsequently sampled video frames identify device A again. Therefore, if the video data corresponding to each inspection task is extracted from the video data at a fixed time length (the same time interval or the same frequency), it is very likely that the multiple sampled video frames will identify multiple duplicate industrial devices, resulting in redundant matching processes, wasted time and resources, and affected efficiency.

[0055] Ideally, for the video data corresponding to a pre-defined inspection task, it would be desirable to sample only N video frames, with one video frame matching one industrial device, where N is the total number of industrial devices to be inspected for the inspection task. However, in reality, it is impossible to determine before sampling whether the industrial device identified in each video frame is a duplicate of an already identified industrial device. Therefore, this embodiment of the present invention optimizes steps S110-S140 to sample as few video frames as possible while ensuring that the expected industrial device can be identified from the video data.

[0056] In general, the maintenance personnel will inspect each industrial equipment according to the sequence of the inspection task, which means that any industrial equipment will be concentrated in the video data of a certain time period and rarely appear in the video data of other time periods. Therefore, the embodiment of the present application provides an optimized sampling strategy for collecting a plurality of video frames, so that after a certain industrial equipment is identified in a video frame, the video frames within a certain time period around the video frame are reduced for re-collection and feature point calculation matching.

[0057] In the preferred embodiment of the present application, step S110 can include: obtaining the total time T of the video data; determining the number N of industrial equipment to be inspected by the inspection task and the unique identifier of each industrial equipment; taking the first frame of the video data as the first sampling point, and determining other sampling points in sequence according to the preset step length, wherein the preset step length = T / (N-1); and collecting the corresponding video frame based on each sampling point.

[0058] For example, the total time T of the video data is obtained, and the number N of industrial equipment to be inspected by the inspection task is determined. Starting from the first frame, the video frames are collected according to the preset step length of T / (N-1), and N video frames can be obtained.

[0059] Preferably, step S130 can further include: for each sampling point, when the corresponding video frame matches an industrial equipment, recording the unique identifier of the sampling point and the corresponding industrial equipment; and when the corresponding video frame does not match an industrial equipment, recording the sampling point as unidentified.

[0060] Based on the above example, for the N video frames collected, the feature point matching calculation is performed according to step S130 to obtain the industrial equipment identified by each video frame. The unique identifier (for example, the equipment name or equipment ID) of the matched industrial equipment can be recorded for each sampling point. When a certain video frame group does not match an industrial equipment, the "unidentified" identifier (for example, 0 is recorded) is recorded for the sampling point.

[0061] Preferably, when the industrial equipment matched by the plurality of video frames does not include all industrial equipment in the preset inspection task, the method for monitoring the inspection of industrial equipment can further include the following steps:

[0062] Based on the above example, all unidentified industrial equipment and the number m of industrial equipment are obtained. When m is not 0, additional sampling is required to determine whether the industrial equipment is unidentified or not inspected.

[0063] Step S10 , searching for two adjacent sampling points that meet the following requirements based on the recorded sampling points: 1) they are marked with unique identifiers of different industrial equipment or are marked as unidentified; 2) the time interval between the two sampling points exceeds a preset time.

[0064] The preset time is the minimum duration t that an industrial device must appear. For example, if two adjacent sampling points P1 and P2 are each marked with an industrial device (e.g., unique identifiers E1 and E2), and the time interval between P1 and P2 exceeds the preset time t, then the above requirement is met. For example, if one of two adjacent sampling points P1 and P2 is marked as unidentified, or both P1 and P2 are marked as unidentified, and the time interval between P1 and P2 exceeds the preset time t, then the above requirement is also met.

[0065] Step S20: Select two adjacent sampling points that meet the requirements and have the largest time interval.

[0066] Among them, if the selected points meet the above requirements 1) and 2), and there are multiple adjacent sampling points with the largest time interval, two adjacent sampling points are randomly selected.

[0067] Step S30 : taking a point between the two selected adjacent sampling points as a new sampling point, and capturing a new video frame based on the new sampling point.

[0068] For example, the midpoint of the two adjacent sampling points selected is used as a new sampling point. As described in step S110, the time interval between the two adjacent sampling points P1 and P2 selected must meet the requirement of exceeding the preset time t. This is because if the interval between P1 and P2 is less than t, then when a new sampling point P is selected between P1 and P2, m , even through the new sampling point P m The captured video frames are matched to the new industrial equipment, but it will also make P m If the time interval between P1 and P2 is less than t, even if a new industrial device is matched, it is not considered that the industrial device has completed the inspection. If the time interval between P1 and P2 (set as t') is greater than t, then there is at least a time interval (t(P1), t(P2) + t'-t), where t' = t(P2) - t(P1), t(P1) and t(P2) represent the moments corresponding to sampling points P1 and P2 respectively. Only by selecting a new sampling point in this time interval can it be possible to detect new industrial devices with an inspection duration exceeding t, that is, it can ensure that the inspection is effectively completed. Therefore, when the time interval between two adjacent sampling points P1 and P2 is less than 2t and greater than t, the moment corresponding to t(P2) - t can be taken as the new sampling point.

[0069] Step S40, by comparing the feature points of the new video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database, the industrial equipment matched by the new video frame is determined.

[0070] According to the feature point matching algorithm shown in steps S120-S130, it is determined whether the new video frame matches a new industrial equipment.

[0071] Step S50, when the new video frame group matches an industrial equipment, the new sampling point and the unique identifier of the corresponding industrial equipment are recorded; when the new video frame does not match an industrial equipment, the new sampling point is recorded as unidentified.

[0072] Step S60, the above steps S10-S50 are re-executed. When m=0, it is determined that the inspection task is completed.

[0073] Preferably, when two adjacent sampling points satisfying the requirement cannot be found, it is determined that the inspection task fails.

[0074] As described above, the time interval between the new sampling point and the adjacent sampling point is required to be greater than t, therefore, when a new sampling point cannot be set up with a time interval greater than the preset time t with the adjacent sampling point, it is determined that the un-matched industrial equipment is not inspected. The inspection task can be reported as not completed, and the un-inspected industrial equipment can also be reported.

[0075] Accordingly, the preferred embodiment of the present application ensures that all industrial equipments needing inspection can be identified from the video data corresponding to the inspection task, and as few video frames as possible are sampled, so that the maintenance personnel can be determined whether to strictly inspect the industrial equipment by less time and calculation cost, and the execution efficiency is higher.

[0076] The embodiment of the present application also provides a control device for monitoring industrial equipment inspection, the control device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to realize the above-mentioned method for monitoring industrial equipment inspection.

[0077] The embodiment of the present application also provides a machine readable storage medium, and the machine readable storage medium stores instructions, and the instructions make the machine execute the above-mentioned method for monitoring industrial equipment inspection.

[0078] Please refer to Figure 2An embodiment of the present invention further provides a device for monitoring industrial equipment inspections. The device includes at least one processor 201, at least one memory 202 connected to the processor 201, and a bus 203. The processor 201 and the memory 202 communicate with each other via the bus 203. The processor 201 is configured to invoke program instructions stored in the memory 202 to execute the aforementioned method for monitoring industrial equipment inspections. The device 20 in this embodiment of the present invention may be a server, a PC, a PAD, a mobile phone, or the like.

[0079] Please refer to Figure 3 An embodiment of the present invention further provides a system for monitoring industrial equipment inspections, the system comprising a camera device 31 and the above-mentioned control device 30 for monitoring industrial equipment inspections.

[0080] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0081] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0085] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0086] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0088] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for monitoring industrial equipment inspection, characterized in that: The method for monitoring industrial equipment inspections comprises: For the video data corresponding to the preset inspection task, multiple video frames are collected; Extracting feature points of each video frame from the plurality of video frames; Determining the industrial equipment matched by each video frame by comparing the feature points of each video frame with a standard feature point set of each industrial equipment in a preset industrial equipment feature database; and When the industrial equipment matched by the multiple video frames includes all the industrial equipment in the preset inspection task, it is determined that the inspection task is completed. The video data corresponding to the preset inspection task is collected by collecting multiple video frames, including: Obtain the total duration T of the video data; Determine the number N of industrial equipment that needs to be inspected for the inspection task and the unique identifier of each industrial equipment; Taking the first frame of the video data as the first sampling point, determining other sampling points in sequence according to a preset step length, where the preset step length = T / (N-1); and Collect the corresponding video frame based on each sampling point; Determining the industrial equipment matched by each video frame further includes: For each sampling point, when the corresponding video frame matches the industrial equipment, record the unique identifier of the sampling point and the corresponding industrial equipment; When the corresponding video frame does not match the industrial equipment, the sampling point is recorded as unrecognized; When the industrial equipment matched by the multiple video frames does not include all the industrial equipment in the preset inspection task, the method for monitoring the inspection of industrial equipment further includes: Based on the recorded sampling points, find two adjacent sampling points that meet the following requirements: Marked with the unique identification of different industrial equipment or marked as unidentified; The time interval between the two sampling points exceeds a preset time; Select two adjacent sampling points that meet the requirements and have the largest time interval; Using a point between the two selected adjacent sampling points as a new sampling point, so as to capture a new video frame based on the new sampling point; Determining the industrial equipment matched by the new video frame by comparing the feature points of the new video frame with the standard feature point set of each industrial equipment in the preset industrial equipment feature database; When the new video frame group matches an industrial device, the new sampling point and the unique identifier of the corresponding industrial device are recorded; When the new video frame does not match the industrial equipment, the new sampling point is recorded as unrecognized; Repeat the above steps.

2. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: Extracting the feature points of each video frame from the plurality of video frames comprises: For each video frame, generate a corresponding image pyramid, and extract a first feature point set of the video frame in the image pyramid; Using a background filtering algorithm, each pixel of the video frame is labeled with a background label or a non-background label; Mapping pixels marked as background labels to the first feature point set to determine the mapped feature points; and The mapped feature points are removed from the first feature point set, and the remaining feature points are determined as feature points extracted from the video frame.

3. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: Before collecting a plurality of video frames of video data corresponding to a preset inspection task, the method for monitoring an inspection of industrial equipment further includes: Creating the industrial equipment feature database; Acquire images of all industrial equipment within a preset range taken from multiple angles; For each piece of industrial equipment, a corresponding image set is constructed based on images taken from multiple angles. Constructing the corresponding standard feature point set based on the image set of the industrial equipment; and The constructed standard feature point set and the corresponding industrial equipment are stored in the industrial equipment feature database.

4. The method for monitoring industrial equipment inspection according to claim 3, characterized in that: For each industrial device, a corresponding image set is constructed based on images taken from multiple angles, including: For each image in the image set, extracting a corresponding second feature point set; By using the background filtering algorithm, each pixel of the image is labeled with a background label or a non-background label; Mapping the pixels marked as background labels to the second feature point set to determine the mapped feature points; removing the mapped feature points from the second feature point set, and determining the remaining feature points as feature points extracted from the image; and The standard feature point set corresponding to the industrial equipment is constructed based on the feature points extracted from each image.

5. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: The step of comparing the feature points of each video frame with a set of standard feature points of each industrial device in a preset industrial device feature database to determine the industrial device matched by each video frame includes: According to the preset inspection task, a standard feature point set corresponding to the industrial equipment to be inspected is obtained from the industrial equipment feature database to form an industrial equipment feature data table; and The industrial equipment matched by each video frame is determined by comparing the feature points of each video frame with the standard feature point set of each industrial equipment in the industrial equipment feature data table.

6. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: The step of comparing the feature points of each video frame with a set of standard feature points of each industrial device in a preset industrial device feature database to determine the industrial device matched by each video frame includes: For each video frame, calculating a first descriptor of a feature point of the video frame; and Traverse the standard feature point set corresponding to each industrial equipment in the preset industrial equipment feature database, For each standard feature point set, calculate the second descriptor of each standard feature point therein, and count the number of standard feature points whose distance between the first descriptor and the second descriptor is less than a first preset threshold. A standard feature point set having the largest number of statistical standard feature points and exceeding a second preset threshold is determined, and the industrial equipment corresponding to the standard feature point set is determined as the industrial equipment matched by the video frame.

7. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: When each video frame has more than a preset number of feature points, determining the industrial equipment matched by each video frame by comparing the feature points of each video frame with a standard feature point set of each industrial equipment in a preset industrial equipment feature database includes: For each video frame, calculating a third descriptor for each feature point of the video frame; and Traverse the standard feature point set corresponding to each industrial equipment in the preset industrial equipment feature database, For each standard feature point set of the sub-standard feature point set, calculate the second descriptor of each standard feature point. Calculating the distance between the third descriptor of each feature point and the second descriptor of each standard feature point in the sub-standard feature point set, When each of the feature points has a standard feature point whose distance is less than a first preset threshold, it is determined that the industrial equipment corresponding to the sub-standard feature point set is the industrial equipment matched by the video frame.

8. The method for monitoring industrial equipment inspection according to claim 1, characterized in that: When two adjacent sampling points that meet the requirements are not found, it is determined that the inspection task has failed.

9. A control device for monitoring industrial equipment inspection, characterized in that: The control device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the method for monitoring industrial equipment inspection according to any one of claims 1 to 8.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores instructions, which enable the machine to execute the method for monitoring industrial equipment inspection according to any one of claims 1-8.

11. A system for monitoring industrial equipment inspection, characterized in that: The system includes a camera device and the control device for monitoring industrial equipment inspection according to claim 9.

Images