A device monitoring method, apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose a device monitoring method, device, apparatus and storage medium. The method comprises: determining feature parameter information of a to-be-monitored device in an original image according to calibration results corresponding to at least two monitoring cameras respectively; determining a first monitoring result of the to-be-monitored device based on the feature parameter information and device parameters; feeding back the first monitoring result to a client, obtaining a corresponding learning result based on the first monitoring result, the device parameters and a preset neural network model, determining a second monitoring result of the to-be-monitored device according to the first monitoring result and the learning result, and feeding back the second monitoring result to the client, so that the client processes the to-be-monitored device according to the second monitoring result. Through the above technical solution, the embodiments of the present application can directly obtain abnormal information when a device fails, so that personnel directly find the key point of the problem, and increase the flexibility and intelligence of real-time monitoring of the device.

Description

Technical Field

[0001] This invention relates to the field of equipment monitoring technology, and in particular to an equipment monitoring method, apparatus, device, and storage medium. Background Technology

[0002] As electronic manufacturing technologies become more advanced, equipment functions more and its integration increases. The number of processes and action types involved in a single piece of equipment is also constantly increasing. However, in the initial stages of introducing entirely new equipment categories, various problems can arise during production, including abnormal actions, malfunctions of core components, and the production of defective products. These problems cannot be completely resolved every time; continuous iteration is required for complete improvement. Equipment stability is a key indicator of equipment reliability, and some specific issues can directly impact customer production quality. These problems are difficult to reproduce, leading to a sharp decline in customer trust. Furthermore, currently, equipment itself can only collect limited information. When a malfunction occurs, it's impossible to directly obtain abnormal information to pinpoint the key issues, even those that seem particularly challenging at present. Therefore, there is an urgent need for a method to monitor equipment status and actions, increasing the accuracy, flexibility, and intelligence of real-time equipment monitoring. Summary of the Invention

[0003] In view of this, the present invention provides a device, apparatus, equipment and storage medium for monitoring equipment, which can directly obtain abnormal information when equipment malfunctions, enabling personnel to directly find the key point of the problem, and increasing the flexibility and intelligence of real-time equipment monitoring.

[0004] According to one aspect of the present invention, an embodiment of the present invention provides a device monitoring method, the method comprising:

[0005] The feature parameter information of the device to be monitored in the original image is determined based on the calibration results of at least two pre-calibrated surveillance cameras, wherein the feature parameter information includes: actual spatial location information and head working status information;

[0006] Based on the actual spatial location information, the head working status information, and the pre-configured device parameters, a first monitoring result for the device to be monitored is determined.

[0007] The first monitoring result is fed back to the client, and the corresponding learning result is obtained based on the first monitoring result, the pre-configured device parameters, and the preset neural network model. The second monitoring result for the device to be monitored is determined according to the first monitoring result and the learning result, and the second monitoring result is fed back to the client so that the client can process the device to be monitored according to the second monitoring result.

[0008] According to another aspect of the present invention, embodiments of the present invention also provide a device monitoring apparatus, the apparatus comprising:

[0009] The information determination module is used to determine the feature parameter information of the device to be monitored in the original image based on the calibration results of at least two types of pre-calibrated surveillance cameras. The feature parameter information includes: actual spatial location information and head working status information.

[0010] The first monitoring module is used to determine the first monitoring result of the device to be monitored based on the actual spatial location information, the head working status information and the pre-configured device parameters.

[0011] The first monitoring module is used to feed back the first monitoring result to the client, and obtain the corresponding learning result based on the first monitoring result, pre-configured device parameters and a preset neural network model. Based on the first monitoring result and the learning result, it determines the second monitoring result for the device to be monitored, and feeds back the second monitoring result to the client so that the client can process the device to be monitored based on the second monitoring result.

[0012] According to another aspect of the present invention, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform any of the device monitoring methods described in the embodiments of the present invention.

[0016] According to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute and implement the device monitoring method described in any embodiment of the present invention.

[0017] The technical solution of this invention determines the actual spatial location information and head working status information of the device to be monitored in the original image through the calibration results of at least two types of monitoring cameras; determines the first monitoring result of the device to be monitored based on the actual spatial location information, head working status information, and pre-configured device parameters; feeds the first monitoring result back to the client, and obtains the corresponding learning result based on the first monitoring result, device parameters, and a preset neural network model; determines the second monitoring result of the device to be monitored based on the first monitoring result and the learning result, and feeds the second monitoring result back to the client. This allows personnel to directly find the key point of the problem when the device malfunctions by directly obtaining abnormal information, increasing the flexibility and intelligence of real-time device monitoring.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a device monitoring method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a single-camera calibration according to an embodiment of the present invention;

[0022] Figure 3 A flowchart illustrating another device monitoring method provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram showing the position of a spatial point in a single set of cameras, provided in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram illustrating the conversion between single-camera calibration and actual position, provided in an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram illustrating the actual status monitoring of a device under surveillance by multiple cameras in different orientations, as provided in an embodiment of the present invention.

[0026] Figure 7This is a schematic diagram of a head status monitoring camera used to monitor the head status of a device under surveillance, as provided in an embodiment of the present invention.

[0027] Figure 8 A schematic diagram of a 3D model of a running trajectory provided in an embodiment of the present invention;

[0028] Figure 9 This is a flowchart illustrating another device monitoring method provided in an embodiment of the present invention;

[0029] Figure 10 This is a schematic architecture diagram illustrating the relationship between a monitoring information module, equipment, and a customer information system, provided in an embodiment of the present invention.

[0030] Figure 11 This is a schematic diagram of the architecture of a device monitoring apparatus according to an embodiment of the present invention;

[0031] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] In one embodiment, Figure 1This is a flowchart of a device monitoring method provided in an embodiment of the present invention. This embodiment is applicable to the situation of real-time monitoring of smart devices. The method can be executed by a device monitoring device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0035] like Figure 1 As shown, the device monitoring method in this embodiment includes the following specific steps:

[0036] S110. Determine the feature parameter information of the device to be monitored in the original image based on the calibration results of at least two pre-calibrated monitoring cameras. The feature parameter information includes: actual spatial location information and head working status information.

[0037] The "at least two types of surveillance cameras" can be understood as two different types of surveillance cameras, including but not limited to at least three sets of equipment status monitoring cameras and a side-mounted equipment working head monitoring camera; of course, each set of surveillance cameras can include surveillance cameras in at least two directions. The equipment to be monitored in this embodiment may include, but is not limited to, a dispensing device.

[0038] In this embodiment, the feature parameter information may include the actual spatial location information and head working status information of the device to be monitored. These can be obtained by combining different types and different numbers of monitoring cameras.

[0039] In this embodiment, each surveillance camera is pre-calibrated, and the calibration result of each surveillance camera includes at least: the camera's intrinsic and extrinsic parameters and coordinate mapping matrix. In some embodiments, at least two camera calibration methods may be used, including: for each surveillance camera, after fixing each camera in place, using a calibration plate mounted on a plane perpendicular to each camera, with the center of the calibration plate vertically coinciding with the center of the optical axis of each camera, to calibrate the surveillance cameras and obtain their intrinsic and extrinsic parameters and coordinate mapping matrix. For example, to facilitate a better understanding of the surveillance camera calibration process, Figure 2 This is a schematic diagram illustrating a single-camera calibration according to an embodiment of the present invention. First, after mounting and fixing the dual cameras, a calibration plate is mounted on a plane perpendicular to the left camera. The center of the calibration plate is aligned vertically with the center of the camera's optical axis. The left camera is then calibrated to obtain its intrinsic and extrinsic parameters and mapping matrix. The right camera is then calibrated in the same manner, followed by image stitching and coordinate transformation. Based on the calibration results, the two original images are corrected. The corrected images are located on the same plane and are parallel to each other. Pixel matching is then performed on the two corrected images.

[0040] In this embodiment, the actual spatial location information and head working status information of the device under monitoring in the original image can be obtained by using the calibration results corresponding to two or more types of pre-calibrated monitoring cameras. In some embodiments, the actual spatial location information of the device under monitoring in the original image can be determined based on the calibration results of at least three sets of status monitoring cameras, and the head working status information of the device under monitoring in the original image can be obtained by using the calibration results corresponding to the head monitoring camera. Of course, this embodiment can also use other monitoring methods to determine the feature parameter information of the device under monitoring in the original image, and this embodiment does not impose any limitations on this.

[0041] S120: Determine the first monitoring result of the device to be monitored based on the actual spatial location information, head working status information and pre-configured equipment parameters.

[0042] The pre-configured device parameters can be understood as the relevant device parameters such as tolerance, type, and range of the actions and states of the device to be monitored, obtained through teaching the device to be monitored. In this embodiment, the pre-configured device parameters may include, but are not limited to, preset device action trajectory information, preset device action tolerance range, preset device state tolerance range, and preset device state type. The first monitoring result refers to the monitoring result obtained by monitoring the actions and working state of the device to be monitored, which may include whether the device to be monitored is in a normal state or an abnormal state.

[0043] In some embodiments, the first monitoring result includes the motion monitoring result of the device under monitoring and the head working status monitoring result; the pre-configured device parameters include at least: preset device motion trajectory information, preset device motion tolerance range, preset device status tolerance range, and preset device status type; in one embodiment, the pre-configuration of device parameters includes: teaching the trajectory points of the device under monitoring to perform device motion, recording the image information corresponding to the current motion position after the device under monitoring reaches the current motion position, obtaining the actual motion position of the device under monitoring based on the image information, continuing to teach the next motion position, and taking the next motion position as the current motion position, returning to record the image information corresponding to the current motion position until the trajectory point teaching ends, and forming trajectory points from each actual motion position, setting the preset device motion tolerance range of the device under monitoring according to the trajectory points; for example, in a conventional dispensing device, the dispensing device mainly uses the motion axis and the dispensing head as the objects of motion status monitoring. The planned motion position of the dispensing device program and the motion of the device itself, such as cleaning glue, discharging glue, and returning to the standby position, are detected. When presetting actions, the device must reach the designated action position. The image, status, and calculated position of this position are then recorded and matched with the program's own position. Note: The device's coordinate system must also be matched with the motion monitoring coordinate system before comparison with the device's own position can be made. For example, if a dispensing device program needs to dispense a straight line, the node positions of the program trajectory must first be taught during action presetting to calculate the detected trajectory position. After matching the coordinate systems, the position is compared with the program's position, and a position offset tolerance is set. This allows for the detection of abnormal motion states when the program position is abnormal.

[0044] In one embodiment, when setting preset device state tolerance ranges and preset device state types for the device to be monitored, a side-mounted working head monitoring camera is used to pre-configure the preset device state tolerance ranges and preset device state types. For the normal state of the working head of the device to be monitored, the side-mounted working head monitoring camera takes pictures to obtain corresponding normal state images, and extracts the normal state device head state image features from these images. For abnormal state of the working head of the device to be monitored, the side-mounted working head monitoring camera takes pictures to obtain corresponding abnormal state images, and extracts the abnormal state device head state image features from these images. Based on the normal state device head state image features and the abnormal state device head state image features, the features and area tolerance of the corresponding working head state of the device to be monitored are set. For example, the head mainly presets various states for the main working heads of the detection equipment, such as mounting heads, dispensing heads, and welding machine heads. Here, we take the dispensing head as an example, setting various states of dispensing adhesive. After setting, abnormal states can be fed back through the communication module. For example, taking dispensing equipment as an example, if the dispensing valve body experiences abnormalities during dispensing, such as blockage, inconsistent dispensing size, residual adhesive sticking to the needle causing abnormal shape, or no dispensing at all, large-scale defects will occur during production. To address these defects, a side camera is used for monitoring and pre-setting state types, such as a no-dispensing state. The pre-setting process involves first taking a picture of the normal dispensing state using the side camera, then selecting the adhesive dispensing from the needle area, setting the detection features and area tolerance, and finally, detecting the dispensing state of the needle.

[0045] In this embodiment, the actual spatial location information can be used to form the trajectory point information of the device to be monitored. The trajectory point information is compared with the preset device action trajectory information to obtain a first comparison result. Based on the first comparison result and the preset device action tolerance range, the action monitoring result of the device to be monitored is determined in response to the head state monitoring signal. The head state image monitored by the side working head monitoring camera is acquired, the image features in the head state image are extracted, and the image features are compared with the preset device state tolerance range to obtain a second comparison result. Based on the second comparison result and the preset device state type, the head state monitoring result of the device to be monitored is determined. It can be understood that the action monitoring camera performs feature recognition on the head of the functional component, captures the position state of the head in real time and compares it with the position set by the program, and obtains the monitoring result based on the comparison result. In some embodiments, the device state transition set can also be obtained by constructing a behavior state model of the automated device, analyzing the action instructions of the automated device control part and its corresponding device actions, and the first monitoring result of the device to be monitored. In some embodiments, the automated device can also be subjected to atomic action judgment, atomic state judgment, state transition judgment and state transition sequence judgment in sequence to diagnose abnormal behavior state of the device. This embodiment does not limit this.

[0046] S130. Feed back the first monitoring result to the client, and obtain the corresponding learning result based on the first monitoring result, the pre-configured device parameters and the preset neural network model. Determine the second monitoring result of the device to be monitored based on the first monitoring result and the learning result, and feed back the second monitoring result to the client so that the client can process the device to be monitored based on the second monitoring result.

[0047] The second monitoring result can be understood as the monitoring result obtained by learning the first monitoring result and preset equipment action trajectory information, preset equipment action tolerance range, preset equipment status tolerance range and preset equipment status type through a neural network model based on the first monitoring result.

[0048] In this embodiment, different operations can be performed based on the monitoring status of the first monitoring result to feed back the data information of various devices under that monitoring status to the client for corresponding display. Specifically, if the first monitoring result is in an abnormal state, the location information, abnormal actions, and corresponding actual detection status of the monitored device under the abnormal state are recorded and fed back to the client. Based on the location information, a 3D simulation trajectory diagram and an actual trajectory diagram of the monitored device are generated and fed back to the client for display, and a prompt or device alarm action is triggered. If the first monitoring result is in a normal state, the device operation status and the compared location information of the monitored device under the normal state are recorded, and a 3D model diagram of the operation trajectory is generated based on the location information and fed back to the client for display. If there are no abnormalities, after each device cycle action, the previously recorded information is stored in the memory, and the latest information is always displayed. Within a preset time period, the first monitoring results in different time periods are compared for each category result to obtain a third comparison result. The third comparison result is fed back to the client as the second monitoring result so that the client can process the monitored device according to the third comparison result.

[0049] The technical solution of this invention determines the actual spatial location information and head working status information of the device to be monitored in the original image through the calibration results of at least two types of monitoring cameras; determines the first monitoring result of the device to be monitored based on the actual spatial location information, head working status information, and pre-configured device parameters; feeds the first monitoring result back to the client, and obtains the corresponding learning result based on the first monitoring result, device parameters, and a preset neural network model; determines the second monitoring result of the device to be monitored based on the first monitoring result and the learning result, and feeds the second monitoring result back to the client. This allows personnel to directly find the key point of the problem when the device malfunctions by directly obtaining abnormal information, increasing the flexibility and intelligence of real-time device monitoring.

[0050] In one embodiment, Figure 3 This is a flowchart of another device monitoring method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment determines the feature parameter information of the device to be monitored in the original image according to the calibration results corresponding to at least two types of pre-calibrated monitoring cameras. Based on the actual spatial location information, head working status information and pre-configured device parameters, a first monitoring result of the device to be monitored is determined. The first monitoring result is fed back to the client, and the corresponding learning result is obtained based on the first monitoring result, the pre-configured device parameters and the preset neural network model. The second monitoring result of the device to be monitored is determined according to the first monitoring result and the learning result, and the second monitoring result is fed back to the client for further refinement.

[0051] like Figure 3 As shown, the device monitoring method in this embodiment may specifically include the following steps:

[0052] S310. Determine the actual spatial location information of the monitored device in the original image based on the first calibration results corresponding to at least three sets of monitoring cameras.

[0053] The actual spatial location information includes the location information of the movement points of the monitored equipment in the actual spatial location.

[0054] In this embodiment, the actual spatial location information of the monitored device in the original image can be determined based on the first calibration results corresponding to at least three groups of monitoring cameras. Specifically, based on the calibration results corresponding to each monitoring camera in each group, image correction and pixel matching are performed on the original image of the monitored device to obtain the location information. The location information is then integrated to obtain the actual spatial location information of the monitored device in the actual spatial coordinate system.

[0055] In some embodiments, determining the actual spatial location information of the monitored device in the original image based on the first calibration results corresponding to at least three sets of monitoring cameras further includes:

[0056] For each group of surveillance cameras, the original image of the device to be monitored is corrected based on the calibration results of each surveillance camera in each group to obtain the corrected target image. Then, the pixel points in the target image are matched to obtain the position information of the device to be monitored in the target image.

[0057] The location information corresponding to each group of monitoring cameras is integrated to obtain the actual spatial location information of the monitored device in the actual spatial coordinate system.

[0058] In this embodiment, for each group of monitoring cameras, image correction is performed on the original image of the device to be monitored based on the calibration results corresponding to each monitoring camera in each group to obtain the corrected target image. Then, pixel matching is performed on each pixel in the target image to obtain the position information of the device to be monitored in the target image. Finally, the position information corresponding to each group of monitoring cameras is integrated to obtain the actual spatial position information of the device to be monitored in the actual spatial coordinate system.

[0059] For example, the position of a single location within a single set of cameras can be calculated by combining the positions of the points observed by that single set of cameras with those positions in the remaining cameras, thus obtaining the specific spatial position in the actual coordinate system. When selecting representative point features, positions that can be observed simultaneously from three directions should be prioritized. Figure 4 This is a schematic diagram illustrating the position of a spatial point within a single set of cameras, as provided in an embodiment of the present invention. Figure 4As shown, in an image captured by the OL camera (where O represents the optical center, L represents left, and R represents right), point P may have multiple actual positions on the left camera, such as P11, P12, P13, P14, and P15. By examining the image captured by the right camera, we can pinpoint which of these five positions (P11, P12, P13, P14, and P15) point P is mapped onto, thus accurately determining point P's position within a single set of camera images. By combining images from multiple sets of cameras and using the same method, the actual spatial position of point P can be obtained.

[0060] In this embodiment, to facilitate the understanding of the corrected image and the conversion and calibration of its actual position, Figure 5 This is a schematic diagram illustrating the conversion between calibration and actual position of a single camera group, provided by an embodiment of the present invention. Specifically, it illustrates the conversion between calibration and actual position of two cameras (left and right) within a single camera group.

[0061] S220. Determine the head working status information of the device to be monitored in the original image based on the second calibration result corresponding to the side working head monitoring camera.

[0062] In this embodiment, the head working status information of the monitored device in the original image can be determined based on the second calibration result corresponding to the side working head monitoring camera. The head working status information includes the head image feature information of the monitored device during operation. In this embodiment, taking a dispensing device as an example, the dispensing valve body may exhibit image features such as blockage, dispensing size, needle tip adhesion leading to abnormal shape, and no dispensing during dispensing operations.

[0063] S230. The actual spatial location information is used to form the trajectory point information of the device to be monitored. The trajectory point information is compared with the preset device motion trajectory information to obtain the first comparison result. Based on the first comparison result and the preset device motion tolerance range, the motion monitoring result of the device to be monitored is determined.

[0064] The first comparison result refers to the comparison result between the trajectory point information and the preset device motion trajectory information. This comparison result may include whether the comparison is within the preset device motion tolerance range or whether the comparison result is outside the preset device motion tolerance range.

[0065] In this embodiment, the preset device action tolerance range can be automatically set through experience or manual intervention, and this embodiment does not impose any restrictions on it.

[0066] In this embodiment, the actual spatial location information can be used to form the trajectory point information of the device to be monitored. The trajectory point information is compared with the preset device motion trajectory information to obtain a first comparison result. Based on the first comparison result and the preset device motion tolerance range, the motion monitoring result of the device to be monitored is determined. Specifically, if the comparison result between the trajectory point information and the preset device motion trajectory information is within the preset device motion tolerance range, the motion monitoring result is considered to be in a normal state; if the comparison result between the trajectory point information and the preset device motion trajectory information is not within the preset device motion tolerance range, the motion monitoring result is considered to be in an abnormal state.

[0067] In this embodiment, to facilitate a better understanding of how multiple sets of cameras in different locations monitor the actual state of the monitored equipment, Figure 6 This is a schematic diagram illustrating the use of multiple cameras at different locations to monitor the actual status of an equipment under surveillance, as provided in an embodiment of the present invention. In this embodiment, the monitoring of the automated equipment under surveillance is illustrated using a dispensing head device as an example.

[0068] S340. In response to the head status monitoring signal, acquire the head status image monitored by the side working head monitoring camera, extract the image features in the head status image, compare the image features with the preset device status tolerance range to obtain the second comparison result, and determine the head status monitoring result of the device to be monitored based on the second comparison result and the preset device status type.

[0069] The second comparison result can be understood as the comparison result between the image features and the preset device state tolerance range. This comparison result can include the image features within the preset device state tolerance range, or the image features within the preset device state tolerance range.

[0070] In this embodiment, in response to the head status monitoring signal, a head status image monitored by a side-mounted head monitoring camera is acquired. Image features are extracted from the head status image, and the image features are compared with a preset device status tolerance range to obtain a second comparison result. The head status monitoring result of the device to be monitored is determined based on the second comparison result and the preset device status type. Specifically, if the image features are within the preset device status tolerance range, the second comparison result is considered normal, and the result is matched with the preset device status type to obtain the head status monitoring result of the device to be monitored. If the image features are not within the preset device status tolerance range, the second comparison result is considered abnormal, and the result is matched with the preset device status type to obtain the abnormal head status monitoring type of the device to be monitored.

[0071] In this embodiment, to facilitate a better understanding of how the head-monitoring camera monitors the actual head status of the device being monitored, Figure 7This is a schematic diagram illustrating a head status monitoring camera used to monitor the head status of a device under monitoring, as provided in an embodiment of the present invention. In this embodiment, the head status monitoring of the automated device under monitoring is illustrated using the dispensing nozzle of a dispensing head device as an example.

[0072] S350. Record the abnormal state and the normal state in the memory.

[0073] In this embodiment, the first monitoring result includes an abnormal monitoring result and a normal monitoring result, and the abnormal status and the normal status are recorded in the memory.

[0074] S360: If the first monitoring result is abnormal, record the location information, abnormal actions, and corresponding actual detection status of the monitored device under abnormal conditions and feed them back to the client. Based on the location information, generate a 3D simulation trajectory map and an actual trajectory map of the monitored device, feed them back to the client for display, and provide prompts or trigger device alarm actions.

[0075] In this embodiment, when an abnormal state is detected in the first monitoring result, the location information, abnormal actions, and corresponding actual detection status of the monitored device under abnormal conditions are recorded and fed back to the client. Based on the location information, a 3D simulation trajectory diagram and an actual trajectory diagram of the monitored device are generated and fed back to the client for display, along with prompts or device alarm actions. This can be understood as timely display and feedback upon detection of an abnormal state, recording it in memory to prevent adverse or serious consequences. The device operation status monitoring system monitors the actual operation of the device. During operation, the monitoring system interacts with the device program in real time to confirm the device status and whether any abnormalities have occurred. For example, the location information, abnormal actions, and actual detection status under abnormal conditions are recorded, and the location information, a 3D simulation trajectory diagram, and an actual trajectory diagram are generated. Communication is then established with the device control module to prompt or trigger device alarm actions. For example, to facilitate a better understanding of the generated 3D model diagram of the monitored device's operating trajectory... Figure 8 This is a schematic diagram of a 3D model of a running trajectory provided in an embodiment of the present invention.

[0076] S370. If the first monitoring result is normal, record the device operation status of the device to be monitored under normal conditions and the location information after comparison. Generate a 3D model of the operation trajectory based on the location information and send it back to the client for display. If there is no abnormality, after each device cycle action, store the previously recorded information in the memory and always display the latest information.

[0077] In this embodiment, if the first monitoring result is normal, the device's operating status and the compared location information are recorded under normal conditions. A 3D model of the operating trajectory is generated based on the location information and fed back to the client for display. If there are no abnormalities, the previously recorded information is stored in the memory after each device cycle, ensuring that the latest information is always displayed. This can be understood as follows: if no abnormalities occur, the device's operating status and the compared location information are recorded, and a 3D model of the operating trajectory is generated. This 3D model can be displayed on the client. If there are no abnormalities, the previously recorded information is stored in the memory after each device cycle, ensuring that the latest information is always displayed.

[0078] S380. Input the first monitoring result and the pre-configured device parameters into the preset neural network model for learning to obtain the corresponding learning result; wherein, the learning result includes the first monitoring result and classification result in different time periods.

[0079] In this embodiment, when a first monitoring result is detected, the first monitoring result and pre-configured device parameters can be input into a preset neural network model for learning to obtain the corresponding learning result; wherein, the learning result includes the first monitoring result and classification result in different time periods.

[0080] S390. Within a preset time period, the first monitoring results in different time periods are compared for each category result to obtain a third comparison result. The third comparison result is then fed back to the client as the second monitoring result so that the client can process the monitored device according to the third comparison result.

[0081] In this embodiment, within a preset time period, the first monitoring results for each category are compared with the first monitoring results within different time periods to obtain a third comparison result. The third comparison result is then fed back to the client as the second monitoring result, allowing the client to process the monitored device based on the third comparison result. For example, the monitored device may have been used for one or two years and requires constant monitoring to determine if maintenance or upkeep is needed. This monitoring involves not only monitoring the operational status but also assessing whether the device has reached the end of its lifespan and requires replacement. Replacement may be necessary if the device's accuracy is insufficient. Similarly, the results of motion monitoring and head status monitoring are stored, such as the comparison results of position information during motion monitoring. This comparison result includes results from different times. For this comparison result, within a certain time period, the deviation between the actual position and the preset position in the comparison result is 0.1mm. After a later period, the deviation is 0.3mm. It is necessary to know from which time period the deviation occurred. This result is also fed back to the customer information system so that the customer can replace the device or perform other operations.

[0082] In this embodiment, deep learning continuously stores monitoring results and preset information, and the monitoring results can be categorized accordingly. During device operation, images captured during device actions are used for training, enabling the detection system to better match the device's motion accuracy characteristics and false alarm rate, resulting in more accurate and intelligent device status judgment. The first main focus is on training the motion monitoring status, primarily learning the device's movement position and actions, which deepens the understanding of device movements and processes. The second main focus is on training the head's state, deepening the training model for various head states. The third main focus is on training the overall device state and accuracy.

[0083] The technical solution of this invention, in accordance with the above embodiments, determines the actual spatial position information of the monitored device in the original image based on the first calibration results corresponding to at least three sets of monitoring cameras, determines the head working state information of the monitored device in the original image based on the second calibration results corresponding to the side working head monitoring camera, forms the trajectory point information of the monitored device from the actual spatial position information, compares the trajectory point information with preset device motion trajectory information to obtain a first comparison result, and determines the motion monitoring result of the monitored device based on the first comparison result and the preset device motion tolerance range; in response to the head state monitoring signal, acquires the head state image monitored by the side working head monitoring camera, extracts image features from the head state image, and sets the image... The system compares the features with the preset device status tolerance range to obtain a second comparison result. Based on the second comparison result and the preset device status type, it determines the head status monitoring result of the device to be monitored and feeds back the first monitoring result to the client. Based on the first monitoring result, the pre-configured device parameters, and the preset neural network model, it obtains the corresponding learning result. Based on the first monitoring result and the learning result, it determines the second monitoring result of the device to be monitored and feeds back the second monitoring result to the client. This allows the client to process the device to be monitored based on the second monitoring result. Furthermore, when a device malfunctions, it can directly obtain abnormal information, enabling personnel to directly find the key point of the problem and increasing the flexibility and intelligence of real-time device monitoring.

[0084] In this embodiment, to facilitate a better understanding of the device monitoring method, Figure 9 This is a flowchart illustrating another device monitoring method according to an embodiment of the present invention. In this embodiment, the operating status can be automatically monitored. When an abnormal state occurs, all relevant data can be recorded in real time, and a visual table and record file can be generated. Furthermore, it can communicate with the device, enabling the device to enter an alarm or alert state when an abnormal phenomenon occurs. It can also automatically generate overall information about the device in the abnormal state, allowing for quick identification of abnormal phenomena and increasing the flexibility and intelligence of real-time device monitoring. Figure 9As shown, the specific implementation method is as follows:

[0085] a1. Perform camera calibration on at least three sets of status monitoring cameras and the side working head monitoring camera to obtain calibration results for multiple sets of cameras.

[0086] a2. Action monitoring status tolerance preset: When presetting actions, the device must reach the specified action position, and the image, status, and converted position of this position must be recorded and matched with the program's own position.

[0087] It should be noted that the coordinate system of the device also needs to be matched with the coordinate system of the motion monitoring system before it can be compared with the device's own position.

[0088] a3. Head status monitoring preset: First, take a picture of the normal glue dispensing state using a side camera. After taking the picture, select the glue dispensing at the needle tip, set the detection features and area tolerance, and the glue dispensing state of the needle tip can be detected after the settings are completed.

[0089] a4. Real-time monitoring by the monitoring system: Based on the calibration results corresponding to at least three sets of monitoring cameras, determine the actual spatial position information of the monitored device in the original image, and based on the second calibration result corresponding to the side working head monitoring camera, determine the head working status information of the monitored device in the original image.

[0090] a5. Determine the first monitoring result of the device to be monitored based on the actual spatial location information, head working status information, and pre-configured equipment parameters.

[0091] a6. Determine if the first monitoring result is in an abnormal state. If yes, execute a7; otherwise, execute a8.

[0092] a7. Record and feed back the location information, abnormal actions, and corresponding actual detection status of the monitored equipment under abnormal conditions to the client. Based on the location information, generate a 3D simulation trajectory map and an actual trajectory map of the monitored equipment, feed them back to the client for display, and provide prompts or trigger equipment alarm actions.

[0093] a8. If the first monitoring result is normal, record the device operation status of the device to be monitored under normal conditions, the location information after comparison, and generate a 3D model of the operation trajectory based on the location information. Feedback to the client for display. If there is no abnormality, after each device cycle action, store the previously recorded information in the memory and always display the latest information.

[0094] In this embodiment, the corresponding learning result can be obtained based on the first monitoring result, the pre-configured device parameters, and the preset neural network model. The second monitoring result of the device to be monitored is determined according to the first monitoring result and the learning result, and the second monitoring result is fed back to the client so that the client can process the device to be monitored according to the second monitoring result.

[0095] In one embodiment, to facilitate a better understanding of the interactive communication diagram between the monitoring information module and the device, and the customer information system, Figure 10 This invention provides a schematic architecture diagram illustrating the relationship between a monitoring information module and equipment, and a customer information system, as shown in one embodiment. Figure 10 As shown, the architecture includes a customer information system, a monitoring information module, and the device to be monitored. The device to be monitored includes a device control module, a device perception module, a device driver module, a device execution module, and a device security module. Through the interaction between the device control module and the monitoring information module, the real-time status monitoring of the device and the device's own status are realized to achieve closed-loop information aggregation, comparison, and information presentation. It can be understood that the first monitoring result is fed back to the client, and the corresponding learning result is obtained based on the first monitoring result, the pre-configured device parameters, and the preset neural network model. The second monitoring result of the device to be monitored is determined according to the first monitoring result and the learning result, and the second monitoring result is fed back to the client so that the client can process the device to be monitored according to the second monitoring result.

[0096] In one embodiment, Figure 11 This is a schematic diagram of the architecture of a device monitoring apparatus according to an embodiment of the present invention. The device monitoring apparatus includes: an information determination module 1110, a first monitoring module 1120, and a second monitoring module 1130.

[0097] The information determination module 1110 is used to determine the feature parameter information of the device to be monitored in the original image based on the calibration results of at least two types of pre-calibrated monitoring cameras. The feature parameter information includes: actual spatial location information and head working status information.

[0098] The first monitoring module 1120 is used to determine the first monitoring result of the device to be monitored based on the actual spatial location information, the head working status information and the pre-configured device parameters.

[0099] The second monitoring module 1130 is used to feed back the first monitoring result to the client, and obtain the corresponding learning result based on the first monitoring result, pre-configured device parameters and preset neural network model, determine the second monitoring result for the device to be monitored according to the first monitoring result and the learning result, and feed back the second monitoring result to the client so that the client can process the device to be monitored according to the second monitoring result.

[0100] In this embodiment, the information determination module determines the actual spatial location information and head working status information of the device to be monitored in the original image through the calibration results corresponding to at least two types of monitoring cameras; the first monitoring module determines the first monitoring result of the device to be monitored based on the actual spatial location information, head working status information, and pre-configured device parameters; the first monitoring result is fed back to the client; and the second monitoring module obtains the corresponding learning result based on the first monitoring result, device parameters, and a preset neural network model, determines the second monitoring result of the device to be monitored based on the first monitoring result and the learning result, and feeds back the second monitoring result to the client. This allows personnel to directly find the key point of the problem when the device malfunctions by directly obtaining abnormal information, increasing the flexibility and intelligence of real-time device monitoring.

[0101] In one embodiment, the calibration result includes at least: intrinsic and extrinsic parameters of the surveillance camera, and coordinate mapping matrix;

[0102] The calibration methods for the at least two surveillance cameras include: for each surveillance camera, after the surveillance camera is installed and fixed, a calibration plate is installed on the vertical plane of each surveillance camera, with the center of the calibration plate coinciding with the center of the optical axis of each surveillance camera, so as to calibrate the surveillance camera and obtain the intrinsic and extrinsic parameters and coordinate mapping matrix of each surveillance camera.

[0103] In one embodiment, the at least two types of monitoring cameras include: at least three sets of status monitoring cameras, and a side-mounted working head monitoring camera; wherein each set of monitoring cameras includes monitoring cameras in at least two directions; correspondingly, the information determination module further includes:

[0104] The first determining unit is used to determine the actual spatial location information of the device to be monitored in the original image based on the first calibration results corresponding to the at least three sets of monitoring cameras, wherein the actual spatial location information includes the motion point information of the device to be monitored in the actual spatial location.

[0105] The second determining unit is used to determine the head working status information of the device to be monitored in the original image based on the second calibration result corresponding to the side working head monitoring camera; wherein, the head working status information includes the head image feature information of the device to be monitored when it is working.

[0106] In one embodiment, the first determining unit further includes:

[0107] The matching subunit is used to perform image correction on the original image of the device to be monitored based on the calibration results corresponding to each monitoring camera in each group to obtain the corrected target image, and to perform pixel matching on each pixel in the target image to obtain the position information of the device to be monitored in the target image.

[0108] The integration subunit is used to integrate the location information corresponding to each group of monitoring cameras to obtain the actual spatial location information of the monitored device in the actual spatial coordinate system.

[0109] In one embodiment, the first monitoring result includes: the motion monitoring result of the device under monitoring and the head working status monitoring result; the pre-configured device parameters include at least: preset device motion trajectory information, preset device motion tolerance range, preset device status tolerance range, and preset device status type; correspondingly, the first monitoring module 1120 further includes:

[0110] The first result determination unit is used to form the trajectory point information of the device to be monitored from the actual spatial location information, compare the trajectory point information with the preset device motion trajectory information to obtain a first comparison result, and determine the motion monitoring result of the device to be monitored based on the first comparison result and the preset device motion tolerance range.

[0111] The second result determination unit is used to respond to the head status monitoring signal, acquire the head status image monitored by the side working head monitoring camera, extract the image features in the head status image, compare the image features with the preset device status tolerance range to obtain a second comparison result, and determine the head status monitoring result of the device to be monitored based on the second comparison result and the preset device status type.

[0112] In one embodiment, the pre-configuration of the device parameters includes: teaching the trajectory points of the device to be monitored to perform device actions; after the device to be monitored reaches the current action position, recording the image information corresponding to the current action position, obtaining the actual action position of the device to be monitored based on the image information, continuing to teach the next action position, and using the next action position as the current action position, returning to record the image information corresponding to the current action position until the trajectory point teaching ends, and forming trajectory points from each of the actual action positions, and setting the preset device action tolerance range of the device to be monitored according to the trajectory points;

[0113] When setting the preset device state tolerance range and preset device state type for the device to be monitored, a side working head monitoring camera is used to pre-configure the preset device state tolerance range and preset device state type. For the normal state of the working head of the device to be monitored, the side working head monitoring camera is used to take pictures to obtain the corresponding normal state image, and the normal state device head state image features are extracted from the normal state image. For the abnormal state of the working head of the device to be monitored, the side working head monitoring camera is used to take pictures to obtain the corresponding abnormal state image, and the abnormal state device head state image features are extracted from the abnormal state image. Based on the normal state device head state image features and the abnormal state device head state image features, the features and area tolerance of the corresponding working head state of the device to be monitored are set.

[0114] In one embodiment, the second monitoring module 1130 further includes:

[0115] A storage unit is used to record the abnormal state and the normal state in the memory;

[0116] The first feedback unit is used to record and feed back to the client the location information, abnormal actions, and corresponding actual detection status of the monitored device under the abnormal state when the first monitoring result is in an abnormal state. It also generates a 3D simulation trajectory map and an actual trajectory map of the monitored device based on the location information, feeds them back to the client for display, and prompts or performs device alarm actions.

[0117] The second feedback unit is used to record the device operation status and the compared location information of the monitored device under the normal state when the first monitoring result is normal, and to generate a 3D model of the operation trajectory based on the location information and feed it back to the client for display. If there is no abnormality, the previously recorded information is stored in the memory after each device cycle action, and the latest information is always displayed.

[0118] In one embodiment, the second monitoring module further includes:

[0119] The third monitoring result subunit is used to input the first monitoring result and the pre-configured device parameters into a preset neural network model for learning to obtain the corresponding learning result; wherein, the learning result includes the first monitoring result and classification result in different time periods;

[0120] The fourth monitoring result subunit is used to compare the first monitoring results in different time periods for each of the classification results within a preset time period to obtain a third comparison result, and to feed the third comparison result back to the client as the second monitoring result so that the client can process the device to be monitored based on the third comparison result.

[0121] In one embodiment, Figure 12 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0122] like Figure 12 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as device monitoring methods.

[0125] In some embodiments, the device monitoring processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the device monitoring method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the device monitoring method by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable device monitoring system, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0131] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0132] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for monitoring equipment, characterized in that, include: The feature parameter information of the device to be monitored in the original image is determined based on the calibration results of at least two pre-calibrated surveillance cameras, wherein the feature parameter information includes: actual spatial location information and head working status information; Based on the actual spatial location information, the head working status information, and the pre-configured device parameters, a first monitoring result for the device to be monitored is determined. The first monitoring result is fed back to the client, and the corresponding learning result is obtained based on the first monitoring result, the pre-configured device parameters, and the preset neural network model. The second monitoring result for the device to be monitored is determined according to the first monitoring result and the learning result, and the second monitoring result is fed back to the client so that the client can process the device to be monitored according to the second monitoring result. The process of obtaining a corresponding learning result based on the first monitoring result, pre-configured device parameters, and a preset neural network model, determining a second monitoring result for the device to be monitored based on the first monitoring result and the learning result, and feeding the second monitoring result back to the client includes: The first monitoring result and the pre-configured device parameters are input into a preset neural network model for learning to obtain the corresponding learning result; wherein, the learning result includes the first monitoring result and classification result in different time periods; Within a preset time period, for each of the classification results, the first monitoring results within different time periods are compared to obtain a third comparison result. The third comparison result is then fed back to the client as the second monitoring result, so that the client can process the device to be monitored based on the third comparison result. The first monitoring result includes: the action monitoring result of the device under monitoring and the head working status monitoring result; The pre-configured device parameters include at least: preset device motion trajectory information, preset device motion tolerance range, preset device status tolerance range, and preset device status type; The determination of the first monitoring result for the device to be monitored based on the actual spatial location information, the head working status information, and pre-configured device parameters includes: The actual spatial location information is used to form the trajectory point information of the device to be monitored. The trajectory point information is compared with the preset device motion trajectory information to obtain a first comparison result. Based on the first comparison result and the preset device motion tolerance range, the motion monitoring result of the device to be monitored is determined. In response to the head status monitoring signal, the head status image monitored by the side working head monitoring camera is acquired, the image features in the head status image are extracted, the image features are compared with the preset device status tolerance range to obtain a second comparison result, and the head status monitoring result of the device to be monitored is determined according to the second comparison result and the preset device status type.

2. The method according to claim 1, characterized in that, The calibration results include at least: the intrinsic and extrinsic parameters of the surveillance camera and the coordinate mapping matrix; The at least two calibration methods for the surveillance cameras include: for each surveillance camera, after each surveillance camera is installed and fixed, a calibration plate is installed on a plane perpendicular to each surveillance camera, with the center vertical position of the calibration plate coinciding with the center position of the optical axis of each surveillance camera, so as to calibrate the surveillance camera and obtain the intrinsic and extrinsic parameters and coordinate mapping matrix of each surveillance camera.

3. The method according to claim 1, characterized in that, The at least two types of monitoring cameras include: at least three sets of status monitoring cameras, and a side-mounted working head monitoring camera; wherein each set of monitoring cameras includes monitoring cameras in at least two directions; correspondingly, the step of determining the feature parameter information of the monitored device in the original image based on the pre-calibrated calibration results corresponding to the at least two types of monitoring cameras further includes: The actual spatial location information of the device to be monitored in the original image is determined based on the first calibration results corresponding to at least three sets of monitoring cameras, wherein the actual spatial location information includes the motion point information of the device to be monitored in the actual spatial location; Based on the second calibration result corresponding to the side-mounted head monitoring camera, the head working status information of the device to be monitored in the original image is determined; wherein, the head working status information includes the head image feature information of the device to be monitored when it is working.

4. The method according to claim 3, characterized in that, The step of determining the actual spatial location information of the monitored device in the original image based on the first calibration results corresponding to the at least three sets of monitoring cameras further includes: For each group of surveillance cameras, the original image of the device to be monitored is corrected based on the calibration results of each surveillance camera in each group to obtain the corrected target image. Then, the position information of the device to be monitored in the target image is obtained by pixel matching of each pixel in the target image. The location information corresponding to each group of monitoring cameras is integrated to obtain the actual spatial location information of the monitored device in the actual spatial coordinate system.

5. The method according to claim 1, characterized in that, The pre-configuration of the device parameters includes: teaching the trajectory points of the device to be monitored to perform device action; after the device to be monitored reaches the current action position, recording the image information corresponding to the current action position, obtaining the actual action position of the device to be monitored based on the image information, continuing to teach the next action position, and using the next action position as the current action position, returning to record the image information corresponding to the current action position until the trajectory point teaching ends, and forming trajectory points from each of the actual action positions, and setting the preset device action tolerance range of the device to be monitored according to the trajectory points; When setting the preset device state tolerance range and preset device state type for the device to be monitored, a side-mounted working head monitoring camera is used to pre-configure the preset device state tolerance range and preset device state type. For the normal state of the working head of the device to be monitored, the side-mounted working head monitoring camera is used to take pictures to obtain the corresponding normal state image, and the normal state device head state image features are extracted from the normal state image. For the abnormal state of the working head of the device to be monitored, the side-mounted working head monitoring camera is used to take pictures to obtain the corresponding abnormal state image, and the abnormal state device head state image features are extracted from the abnormal state image. Based on the normal state device head state image features and the abnormal state device head state image features, the features and area tolerance of the corresponding working head state of the device to be monitored are set.

6. The method according to claim 1, characterized in that, The first monitoring result includes monitoring results that are in an abnormal state and monitoring results that are in a normal state; The step of feeding back the first monitoring result to the client further includes: The abnormal state and the normal state are recorded in the memory; If the first monitoring result is in an abnormal state, the location information, abnormal actions, and corresponding actual detection status of the monitored device under the abnormal state are recorded and fed back to the client. Based on the location information, a 3D simulation trajectory map and an actual trajectory map of the monitored device are generated and fed back to the client for display, and a prompt or device alarm action is performed. If the first monitoring result is normal, the device operation status and the location information of the monitored device under normal status are recorded, and a 3D model of the operation trajectory is generated based on the location information and fed back to the client for display. If there is no abnormality, the previously recorded information is stored in the memory after each device cycle action, and the latest information is always displayed.

7. A device for monitoring equipment, characterized in that, include: The information determination module is used to determine the feature parameter information of the device to be monitored in the original image based on the calibration results of at least two types of pre-calibrated surveillance cameras. The feature parameter information includes: actual spatial location information and head working status information. The first monitoring module is used to determine the first monitoring result of the device to be monitored based on the actual spatial location information, the head working status information and the pre-configured device parameters. The second monitoring module is used to feed back the first monitoring result to the client, and obtain the corresponding learning result based on the first monitoring result, pre-configured device parameters and preset neural network model, determine the second monitoring result for the device to be monitored according to the first monitoring result and the learning result, and feed back the second monitoring result to the client so that the client can process the device to be monitored according to the second monitoring result; The second monitoring module includes: The third monitoring result subunit is used to input the first monitoring result and the pre-configured device parameters into a preset neural network model for learning to obtain the corresponding learning result; wherein, the learning result includes the first monitoring result and classification result in different time periods; The fourth monitoring result subunit is used to compare the first monitoring results in different time periods for each of the classification results within a preset time period to obtain a third comparison result, and to feed the third comparison result back to the client as the second monitoring result so that the client can process the device to be monitored based on the third comparison result. The first monitoring result includes: the action monitoring result of the device under monitoring and the head working status monitoring result; The pre-configured device parameters include at least: preset device motion trajectory information, preset device motion tolerance range, preset device status tolerance range, and preset device status type; The first monitoring module includes: The first result determination unit is used to form the trajectory point information of the device to be monitored from the actual spatial location information, compare the trajectory point information with the preset device motion trajectory information to obtain a first comparison result, and determine the motion monitoring result of the device to be monitored based on the first comparison result and the preset device motion tolerance range. The second result determination unit is used to respond to the head status monitoring signal, acquire the head status image monitored by the side working head monitoring camera, extract the image features in the head status image, compare the image features with the preset device status tolerance range to obtain a second comparison result, and determine the head status monitoring result of the device to be monitored based on the second comparison result and the preset device status type.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the device monitoring method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the device monitoring method according to any one of claims 1-6.

Images