Abnormal motion state detection method, terminal device and computer-readable storage medium

Abstract

The present application discloses a method for detecting an abnormal motion state, a terminal device, and a computer-readable storage medium. The method comprises: segmenting a conveyor belt region and target device regions from an image to be detected; the conveyor belt region includes a conveyor belt, and the target device region includes a device used to transport the conveyor belt; and determining whether the conveyor belt is in an abnormal motion state based on the distribution relationship between the conveyor belt region and the target device regions. The abnormal motion state detection method of the present application reduces the amount of hardware required, is simple and easy to implement, and effectively improves the applicability of the abnormal motion state detection method in actual production environments by segmenting the conveyor belt region and target device regions from the image to be detected, and determining whether the belt is in an abnormal motion state based on the distribution relationship between the conveyor belt region and the target device regions.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an abnormal motion state detection method, a terminal device, and a computer-readable storage medium. Background Art

[0002] Belt conveyors are continuous conveying machines that use conveyor belts to transport materials. They are widely used in industries such as coal mines, power plants, ports, and metallurgy. Belt deviation is the most common fault during operation. Long-term belt deviation can seriously damage the belt's service life. Severe belt deviation can even cause derailment, leading to significant material tilt and severely impacting industrial production efficiency.

[0003] In the existing technology, the method for detecting the abnormal movement state of the conveyor belt mainly uses a laser transmitter to detect the laser irradiated on the conveyor belt or roller or uses laser ranging to realize conveyor belt abnormality detection. Although this detection method can determine the specific deviation value of the conveyor belt, its detection equipment is complex and it is not possible to set corresponding alarms according to the degree of deviation of the conveyor belt. It is easy to cause false detection and multiple alarms, and its applicability is not strong. Summary of the Invention

[0004] The present application provides an abnormal motion state detection method, a terminal device, and a computer-readable storage medium to solve the technical problem that the abnormal motion state detection method in the prior art is not highly applicable.

[0005] To solve the above technical problems, the first technical solution provided by this application is to provide a method for detecting abnormal motion state, which includes:

[0006] Segmenting a conveyor belt area and various target device areas from the image to be detected; the conveyor belt area includes the conveyor belt, and the target device area includes devices for transporting the conveyor belt;

[0007] Based on the distribution relationship between the conveyor belt area and each of the target device areas, it is determined whether the conveyor belt is in an abnormal movement state.

[0008] In order to solve the above technical problems, the second technical solution provided by the present application is: providing a terminal device, the terminal device comprising a processor and a memory connected to the processor, wherein:

[0009] The memory stores program instructions;

[0010] The processor is used to execute the program instructions stored in the memory to implement the abnormal motion state detection method as described above.

[0011] In order to solve the above technical problems, the third technical solution provided by the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, the abnormal motion state detection method as described above is performed.

[0012] In the abnormal motion state detection method provided by the present application, a terminal device segments a conveyor belt area and various target device areas from an image to be detected; the conveyor belt area includes the conveyor belt, and the target device area includes the equipment used to transport the conveyor belt; based on the distribution relationship between the conveyor belt area and the various target device areas, it is determined whether the conveyor belt is in an abnormal motion state. The abnormal motion state detection method of the present application reduces the amount of hardware by segmenting the conveyor belt area and various target device areas from the image to be detected, and judging whether the belt is in an abnormal motion state based on the distribution relationship between the conveyor belt area and the various target device areas. The method is simple and easy to implement, effectively improving the applicability of the abnormal motion state detection method in actual production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0014] Figure 1 This is a flow chart of the first embodiment of the abnormal motion state detection method provided by the present application;

[0015] Figure 2 This is a flow chart of the second embodiment of the abnormal motion state detection method provided by the present application;

[0016] Figure 3 This is a schematic diagram of a detection interface of an embodiment of a terminal device provided by this application;

[0017] Figure 4 This is a flow chart of the third embodiment of the abnormal motion state detection method provided by the present application;

[0018] Figure 5 This is a flowchart of a fourth embodiment of the abnormal motion state detection method provided by the present application;

[0019] Figure 6 This is a flowchart of the fifth embodiment of the abnormal motion state detection method provided by the present application;

[0020] Figure 7 This is a flowchart of a sixth embodiment of the abnormal motion state detection method provided by the present application;

[0021] Figure 8 This is a schematic diagram of an alarm state according to an embodiment of the abnormal motion state detection method provided by the present application;

[0022] Figure 9 This is a flowchart of the seventh embodiment of the abnormal motion state detection method provided by the present application;

[0023] Figure 10 This is a schematic diagram of a detection interface of another embodiment of a terminal device provided by the present application;

[0024] Figure 11 This is a schematic structural diagram of an embodiment of a terminal device provided by this application;

[0025] Figure 12 It is a structural diagram of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0026] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some, rather than all, structures related to the present application are shown in the accompanying drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] The terms "first," "second," and the like in this application are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "comprising," and "provided with," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0028] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] In order to solve the technical problem that the abnormal motion state detection method in the prior art is not very applicable, the present application proposes an abnormal motion state detection method, which can be run in a browser or application, and can be specifically applied to a terminal device. Among them, the terminal device of the present application can be a server, or it can be a system composed of a server and a local terminal that cooperate with each other. Accordingly, the various parts included in the terminal device, such as various units, subunits, modules, and submodules, can all be set in the server, or can be set separately in the server and the local terminal.

[0030] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here. In some possible implementations, the abnormal motion state detection method of the embodiment of the present application can be implemented by a processor calling computer-readable instructions stored in a memory.

[0031] See Figure 1 and Figure 2 , Figure 1 This is a flow chart of the first embodiment of the abnormal motion state detection method provided by this application. Figure 2 This is a schematic diagram of the detection interface of an embodiment of the terminal device provided by this application. Figure 1 As shown, the steps of the abnormal motion state detection method in the embodiment of the present application are as follows:

[0032] S11: Segmenting the conveyor belt area 20 and various target device areas from the image to be detected; the conveyor belt area 20 includes the conveyor belt, and the target device area includes devices used to transport the conveyor belt.

[0033] In the embodiments of the present application, the image to be inspected can be obtained by capturing the relevant area using an image acquisition device. The image acquisition device may include, but is not limited to, one or more of a box camera, a dome camera, a panoramic camera, an industrial camera, etc. Any one or more of these devices is used to monitor the conveyor belt conveying area, and the image to be inspected is captured periodically in the conveyor belt conveying area.

[0034] As an embodiment, the user can customize a regular area and determine a conveying area of ​​the conveyor belt where the conveyor belt is prone to deviation as a detection area; or set a detection area in each conveyor belt conveying process to collect images to be detected in the detection area, and then detect whether the conveyor belt is deviating.

[0035] Specifically, the camera can be installed top-mounted or diagonally, and the staff can use different installation methods according to the actual scene. The standard for installing the camera is to require the camera to capture the conveyor belt and rollers as much as possible, and the conveyor belt should be as symmetrical as possible in the image to be detected, so that the rollers on both sides of the conveyor belt are as equal as possible. For example, the camera can be installed in the middle of the conveyor belt, such as Figure 2 As shown, the conveyor belt captured by the camera can be in the middle of the image and be axially symmetrical, ensuring that the conveyor belt will not exceed the range of the image to be detected when the conveyor belt deviates seriously.

[0036] When the terminal device detects whether the conveyor belt is deviating, the terminal device obtains the image to be detected captured by the camera and divides the image to be detected into a conveyor belt area 20 and various target device areas; the conveyor belt area 20 includes the conveyor belt, and the target device area includes the equipment used to transmit the conveyor belt.

[0037] S12: Based on the distribution relationship between the conveyor belt area 20 and each target device area, determine whether the conveyor belt is in an abnormal movement state.

[0038] The terminal device obtains the distribution relationship between the conveyor belt area 20 and each target device area of ​​the image to be detected, and determines whether the conveyor belt is in an abnormal movement state based on the distribution relationship between the conveyor belt area 20 and each target device area.

[0039] In an embodiment of the present application, a terminal device segments a conveyor belt region 20 and various target device regions from an image to be detected; the conveyor belt region 20 includes the conveyor belt, and the target device region includes the device used to transport the conveyor belt; based on the distribution relationship between the conveyor belt region 20 and the various target device regions, a determination is made as to whether the conveyor belt is in an abnormal motion state. The abnormal motion state detection method of this embodiment reduces the amount of hardware required, is simple and easy to implement, and effectively improves the applicability of the abnormal motion state detection method in actual production environments by segmenting the conveyor belt region 20 and the various target device regions from the image to be detected, and determining whether the belt is in an abnormal motion state based on the distribution relationship between the conveyor belt region 20 and the various target device regions.

[0040] Optionally, step S12 further includes the following steps: determining a comprehensive deviation parameter based on the distribution relationship and the area of ​​each target device area; and determining whether the conveyor belt is in an abnormal movement state according to the size of the comprehensive deviation parameter and the first deviation threshold.

[0041] The terminal device obtains the distribution relationship between the conveyor belt area and each target device area of ​​the image to be detected. Based on the distribution relationship and the area of ​​each target device area, the number and area of ​​the target device area can be obtained to calculate the comprehensive deviation parameter, and according to the size of the comprehensive deviation parameter and the first deviation threshold, it is determined whether the conveyor belt is in an abnormal movement state.

[0042] For details, see Figure 3 , Figure 3 FIG. 1 is a flow chart of the second embodiment of the abnormal motion state detection method provided by the present application. In the second embodiment, Figure 3 As shown, the steps of the abnormal motion state detection method are as follows:

[0043] S101: Identify a conveyor belt area 20 and several roller detection frames 30 in an image to be detected.

[0044] In this embodiment, the target device for transmitting the conveyor belt in the image to be detected includes rollers, and each target device area of ​​the image to be detected includes a roller detection frame 30, and the number of roller detection frames 30 corresponds to the target device area.

[0045] When the terminal device detects that the conveyor belt is running off track, the terminal device obtains the image to be detected captured by the camera and divides the image to be detected into several areas. Figure 2 As shown, the image to be inspected is divided into a preset area 10, a conveyor belt area 20, a roller detection frame 30, and a circumscribed rectangular area 40. When the image to be inspected is input into the terminal device, the terminal device configures the preset area 10, which is roughly symmetrical about the conveyor belt. Based on the preset area 10, the terminal device generates the circumscribed rectangular area 40. The terminal device is equipped with a first model and a second model. The first model is used to identify the conveyor belt area 20 of the image to be inspected, and the second model is used to identify the roller detection frame 30 of the image to be inspected.

[0046] S102: Acquire a first area of ​​the roller detection frame 30 on one side of the conveyor belt area 20 and a second area of ​​the roller detection frame 30 on the other side of the conveyor belt area 20, wherein the first area is less than or equal to the second area.

[0047] The terminal device obtains a first area of ​​the roller detection frame 30 on one side of the conveyor belt area 20 and a second area of ​​the roller detection frame 30 on the other side of the conveyor belt area 20. The area of ​​the roller detection frame 30 is the area of ​​a rectangular detection frame, and the first area is less than or equal to the second area. In the initial position, that is, when the conveyor belt is not deviating, the first area and the second area are equal, or the ratio of the first area to the second area is equal to 1. When the conveyor belt begins to deviate, the first area is smaller than the second area, and the conveyor belt deviates toward the roller on the side of the conveyor belt area 20 with the first area.

[0048] S103 : Acquire the number of roller detection frames 30 and the area of ​​the background region excluding the conveyor belt region 20 and the roller detection frames 30 .

[0049] Specifically, the background area is the area of ​​the circumscribed rectangular area 40 excluding the conveyor belt area 20. The terminal device obtains the number of roller detection frames 30 and obtains the area of ​​the background area excluding the conveyor belt area 20 and the roller detection frame 30 by calculating pixel values.

[0050] S104: Determine a comprehensive deviation parameter based on the background area, the number of roller detection frames 30 , the first area, and / or the second area.

[0051] After the terminal device obtains the background area, the number of roller detection frames 30, the first area and / or the second area, it determines the comprehensive deviation parameters based on the background area, the number of roller detection frames 30, the first area and / or the second area.

[0052] S105: Determine whether the deviation comprehensive parameter is greater than or equal to the deviation threshold.

[0053] After determining the comprehensive deviation parameter, the terminal device determines whether the comprehensive deviation parameter is greater than or equal to a deviation threshold. The deviation threshold is a threshold preset by the operator based on the conveyor model, specifications, and other parameters. If the comprehensive deviation parameter is greater than or equal to the deviation threshold, the process proceeds to step S106.

[0054] S106: Confirm that the conveyor belt is in an abnormal motion state.

[0055] When the terminal device determines that the comprehensive deviation parameter is greater than or equal to the deviation threshold, it confirms that the conveyor belt is in an abnormal movement state.

[0056] In an embodiment of the present application, a terminal device identifies a conveyor belt area 20 and several roller detection frames 30 in an image to be detected; obtains a first area of ​​the roller detection frame 30 on one side of the conveyor belt area 20 and a second area of ​​the roller detection frame 30 on the other side of the conveyor belt area 20, wherein the first area is less than or equal to the second area; obtains the number of roller detection frames 30 and the area of ​​the background area excluding the conveyor belt area 20 and the roller detection frames 30; determines a comprehensive deviation parameter based on the area of ​​the background area, the number of roller detection frames 30, the first area, and / or the second area; determines whether the comprehensive deviation parameter is greater than or equal to a deviation threshold; and confirms that the conveyor belt is in an abnormal motion state. The abnormal motion state detection method of this embodiment determines the comprehensive deviation parameter by obtaining the background area, the number of roller detection frames 30, the first area, and / or the second area, and then determines the degree of deviation of the conveyor belt, thereby improving the accuracy of conveyor belt abnormality detection; by obtaining the image to be detected to perform conveyor belt abnormality detection, the amount of hardware is reduced, the method is simple and easy to implement, and effectively improves the applicability of the abnormal motion state detection method in actual production environments.

[0057] Furthermore, the abnormal motion state includes multiple deviation states of different levels. After step S106, the abnormal motion state detection method also includes: determining the deviation comprehensive parameter range of the deviation comprehensive parameter mapping based on the deviation comprehensive parameter range mapped by different deviation states among the multiple deviation states; and determining the deviation state mapped by the determined deviation comprehensive parameter range as the motion state of the conveyor belt.

[0058] Specifically, the deviation threshold may correspond to multiple thresholds to classify abnormal motion states into multiple different levels of deviation states. Each deviation state is mapped to a corresponding deviation comprehensive parameter range. After determining the deviation state mapped to the deviation comprehensive parameter range, the deviation state of the image to be detected can be determined by determining the parameter range of the deviation comprehensive parameter of the image to be detected.

[0059] For example, the abnormal motion state can be divided into a mild deviation state and a severe deviation state, with the deviation thresholds including a first deviation threshold and a second deviation threshold, wherein the first deviation threshold is less than the second deviation threshold. When the deviation comprehensive parameter is less than the first deviation threshold, the conveyor belt is confirmed to be in a normal state; when the deviation comprehensive parameter is greater than or equal to the first deviation threshold and less than the second deviation threshold, the conveyor belt is confirmed to be in a mild deviation state; and when the deviation comprehensive parameter is greater than or equal to the second deviation threshold, the conveyor belt is confirmed to be in a severe deviation state. By classifying the conveyor belt's deviation state into a mild deviation state and a severe deviation state, a terminal device can determine whether the conveyor belt is in a severe deviation state based on the set thresholds and then take appropriate warning measures, thereby improving the efficiency of the abnormal motion state detection method. Multiple deviation thresholds can also be set to classify the abnormal motion state into multiple different levels of deviation states. The deviation thresholds and deviation states are not specifically limited here.

[0060] See Figure 4 , Figure 4 This is a flow chart of the third embodiment of the abnormal motion state detection method provided by this application. Figure 4 As shown, step S104 further includes the following steps:

[0061] S201: Obtain background deviation parameters based on the background area.

[0062] like Figure 2 As shown, the background area is the area of ​​the circumscribed rectangular area 40 of the preset area 10 excluding the conveyor belt area 20. Based on the background area obtained by the terminal device, the terminal device divides the background area into a background area on one side of the conveyor belt area 20 and a background area on the other side of the conveyor belt area 20. Based on the area of ​​the background areas on both sides of the conveyor belt area 20, the terminal device calculates the background deviation parameter as shown in the following formula:

[0063]

[0064] Among them, R b is the background deviation parameter, S b1 is the background area on one side of the conveyor belt area 20, S b2 is the background area on the other side of the conveyor belt area 20.

[0065] S202: Based on the number of the roller detection frames 30, obtain the number deviation parameter.

[0066] Based on the number of roller detection frames 30 on both sides obtained by the terminal device, the terminal device calculates the number deviation parameter as shown in the following formula:

[0067]

[0068] Among them, R N is a number deviation parameter, N1 is the number of roller detection frames 30 on one side of the conveyor belt area 20, and N2 is the number of roller detection frames 30 on the other side of the conveyor belt area 20.

[0069] S203: Acquire an area deviation parameter based on the first area and the second area.

[0070] Based on the first area and the second area obtained by the terminal device, the terminal device calculates the area deviation parameter S r .

[0071] S204: Determine a comprehensive deviation parameter based on the background deviation parameter, the quantity deviation parameter, and the area deviation parameter.

[0072] After the terminal device obtains the background deviation parameter, the number deviation parameter and the area deviation parameter, it performs a weighted summation of the background deviation parameter, the number deviation parameter and the area deviation parameter based on the preset weight to obtain the comprehensive deviation parameter.

[0073] Optionally, the terminal device can obtain an area deviation parameter based on the first area and the second area, and determine a comprehensive deviation parameter based on the area deviation parameter; it can also obtain an area deviation parameter based on the first area and the second area, obtain a background deviation parameter based on the background area, and determine a comprehensive deviation parameter based on the area deviation parameter and the background deviation parameter; it can also determine a comprehensive deviation parameter based on at least one of a quantity deviation parameter, an area deviation parameter, and a background deviation parameter.

[0074] Optionally, the preset weight may be as follows:

[0075] R o =0.5×R N +0.3×R b +0.2×R r ;

[0076] Among them, R o is the comprehensive deviation factor, R N is the quantity deviation parameter, R b is the background deviation parameter, R r In a specific implementation, the staff can set the corresponding weight according to the model, specifications and other parameters of the conveyor belt conveyor, and the preset weight is not specifically limited here.

[0077] Obtain the comprehensive deviation factor R o Then, the comprehensive deviation factor R o Mapped to the range of 0-1, the comprehensive deviation parameter is obtained as shown in the following formula:

[0078]

[0079] Among them, m is the comprehensive parameter of deviation.

[0080] In this embodiment of the present application, the terminal device obtains a background deviation parameter based on the background area; a quantity deviation parameter based on the number of roller detection frames 30; and an area deviation parameter based on the first and second areas. The background deviation parameter, quantity deviation parameter, and area deviation parameter are weighted and summed according to preset weights to obtain a comprehensive deviation parameter. The method of this embodiment combines parameters such as the background area, the number of roller detection frames 30, and the first and / or second areas to determine the comprehensive deviation parameter, thereby reducing false detections by the terminal device and effectively improving the accuracy of the conveyor belt detection method.

[0081] Furthermore, step S203 includes the following steps: obtaining the total area of ​​the first area and the second area; and determining an area deviation parameter based on a ratio of the first area to the total area.

[0082] After obtaining the total area of ​​the first area and the second area, the terminal device determines the area deviation parameter based on the ratio of the first area to the total area, as shown in the following formula:

[0083]

[0084] Among them, R r is the area deviation parameter, S r1 is the first area of ​​the roller detection frame 30 on one side of the conveyor belt area 20, S rf It is the total area of ​​the first area and the second area of ​​the roller detection frame 30 on both sides of the conveyor belt area 20.

[0085] See Figure 5 , Figure 5 This is a flow chart of the fourth embodiment of the abnormal motion state detection method provided by this application. Figure 5As shown, in the third embodiment, the abnormal motion state includes the deviation state and the run-away state. After step S106, the abnormal motion state detection method further detects whether the conveyor belt runs out of the range of the roller, including the following steps:

[0086] S301 : Acquire the total area of ​​the conveyor belt area 20 and the partial area within the preset area 10 .

[0087] The terminal device obtains the total area of ​​the conveyor belt area 20 and the partial area of ​​the conveyor belt area 20 located in the preset area 10 .

[0088] S302: Determine comprehensive parameters of the runaway based on the partial area and the total area.

[0089] Based on the total area of ​​the conveyor belt area 20 and the area of ​​the portion of the conveyor belt area 20 located within the preset area 10, the terminal device can obtain the area of ​​the portion of the conveyor belt area 20 located within the preset area 10 and another area of ​​the portion located outside the preset area 10, and determine the comprehensive runaway parameter, as shown in the following formula:

[0090]

[0091] Among them, n is the comprehensive parameter of running and flying; S in S is the area of ​​the conveyor belt area 20 located in the preset area 10; out It is another part of the area of ​​the conveyor belt area 20 located outside the preset area 10.

[0092] S303: Determine whether the comprehensive running-away parameter is greater than or equal to the running-away threshold.

[0093] After obtaining the flight-related comprehensive parameter, the terminal device determines whether the flight-related comprehensive parameter is greater than or equal to the flight-related threshold. If the flight-related comprehensive parameter is greater than or equal to the flight-related threshold, the process proceeds to step S304; if the flight-related comprehensive parameter is less than the flight-related threshold, the process proceeds to step S305.

[0094] S304: Confirm that the conveyor belt is in a deviation state and output a deviation alarm.

[0095] When the terminal device determines that the comprehensive parameter of the runaway is greater than or equal to the runaway threshold, the terminal device confirms that the conveyor belt is still in the preset area 10, the conveyor belt is in a deviation state, and outputs a deviation alarm.

[0096] S305: Confirm that the conveyor belt is in a runaway state and output a runaway alarm.

[0097] When the terminal device determines that the comprehensive parameter of the runaway is less than the runaway threshold, the terminal device confirms that the conveyor belt is outside the preset area 10, that is, the conveyor belt runs out of the roller and is in a runaway state, and outputs a runaway alarm.

[0098] In an embodiment of the present application, a terminal device obtains the total area of ​​the conveyor belt area 20 and the partial area within the preset area 10; determines a runaway comprehensive parameter based on the partial area and the total area; determines whether the runaway comprehensive parameter is greater than or equal to a runaway threshold; if so, determines that the conveyor belt is in a runaway state and outputs a runaway alarm; if not, determines that the conveyor belt is in a runaway state and outputs a runaway alarm. The abnormal motion state detection method of this embodiment calculates the runaway comprehensive parameter to determine whether the conveyor belt is in a runaway state and outputs a runaway alarm. This allows workers to adjust the conveyor belt in a timely manner based on the runaway alarm, reducing production accidents and effectively improving the applicability of the abnormal motion state detection method in actual production environments.

[0099] See Figure 6 , Figure 6 This is a flow chart of the fifth embodiment of the abnormal motion state detection method provided by this application. Figure 6 As shown, in this embodiment, an alarm state machine is used to alarm the conveyor belt in the deviation state. After step S106, the abnormal motion state detection method further includes the following steps:

[0100] S401: Pre-set the alarm time interval.

[0101] The alarm state machine is pre-set with an alarm time interval.

[0102] S402: When it is confirmed that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time starts to be accumulated.

[0103] When the terminal device confirms that the conveyor belt is in an abnormal motion state, it sends the abnormal state information to the alarm state machine. After receiving the abnormal state information, the alarm state machine initializes the alarm time, starts timing, and accumulates the alarm time.

[0104] S403: When the accumulated alarm time reaches the alarm time interval during the duration of confirming that the conveyor belt is in an abnormal motion state, an alarm signal is issued and the alarm time is reset.

[0105] When the accumulated alarm time reaches the alarm interval, the alarm state machine retrieves the conveyor belt status information to confirm whether the conveyor belt is in an abnormal motion state. If the conveyor belt is still in an abnormal motion state, the alarm state machine issues an alarm signal and resets the alarm time to continue accumulating the alarm time for the next phase of monitoring and alarming. If the alarm state machine receives normal status information after the accumulated alarm time reaches the alarm interval, the alarm state machine returns to normal state until it receives abnormal status information again, at which point it initializes the alarm time.

[0106] In this embodiment, an alarm time interval is pre-set; upon confirmation that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time begins to accumulate; within the duration of the confirmed abnormal motion state, when the accumulated alarm time reaches the alarm time interval, an alarm signal is issued and the alarm time is reset. Through the method of this embodiment, the alarm state machine can repeatedly confirm whether the conveyor belt is in an abnormal motion state at a preset interval, avoiding false alarms of conveyor belt deviation and improving the efficiency of the conveyor belt detection method. Because the alarm is not issued until the alarm time reaches the alarm time interval, frequent alarms of conveyor belt deviation are avoided, improving the user experience of conveyor belt detection.

[0107] See Figure 7-8 , Figure 7 This is a flow chart of the sixth embodiment of the abnormal motion state detection method provided by this application. Figure 8 This is a schematic diagram of the alarm state of an embodiment of the abnormal motion state detection method provided by this application. Figure 7 As shown, in this embodiment, the alarm time interval includes a pre-alarm time interval and a repeated alarm time interval; after step S106, the abnormal motion state detection method further includes the following steps:

[0108] S501: When it is confirmed that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time begins to accumulate.

[0109] When the terminal device confirms that the conveyor belt is in an abnormal motion state, it sends the abnormal state information to the alarm state machine. After the alarm state machine receives the abnormal state information of the conveyor belt, the alarm state machine enters the pre-alarm stage, initializes the alarm time, and starts timing to accumulate the alarm time.

[0110] S502: When the accumulated alarm time reaches the pre-alarm time interval, it is determined whether the conveyor belt is in a deviation state.

[0111] In this embodiment, the alarm interval includes a pre-alarm interval and a re-alarm interval. When the accumulated alarm time reaches the pre-alarm interval, the terminal device detects whether the conveyor belt is in an abnormal motion state. If the alarm state machine receives abnormal state information, the process proceeds to step S503. If the alarm state machine receives normal state information, the process proceeds to step S504.

[0112] S503: Continue to accumulate the alarm time until the repeated alarm time interval is reached, and issue an alarm signal.

[0113] When the accumulated alarm time reaches the pre-alarm interval, the alarm state machine receives information about the abnormal movement of the conveyor belt. The alarm state machine then enters the repeated alarm phase, continuing to accumulate the alarm time until the accumulated alarm time reaches the repeated alarm interval. At this point, the terminal device continues to monitor whether the conveyor belt is in abnormal movement. If the alarm state machine receives information about an abnormal state, it issues an alarm signal and resets the alarm time, continuing to accumulate the alarm time and monitor the alarm in the repeated alarm phase. If the alarm state machine receives information about a normal state, it returns to normal.

[0114] S504: Reset the alarm time.

[0115] When the accumulated alarm time reaches the pre-alarm time interval, the alarm state machine receives normal state information, and the alarm state machine resets the alarm time and returns to the normal state.

[0116] In this embodiment, upon confirming that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time begins to accumulate. When the accumulated alarm time reaches the pre-alarm interval, the conveyor belt is detected to determine whether it is in an abnormal motion state. The alarm time continues to accumulate until the repeated alarm interval is reached, at which point an alarm signal is issued. The alarm time is then reset. Through the method of this embodiment, the alarm state machine can preset intervals to enter different alarm stages after the alarm time has accumulated. Based on the different alarm stages, repeated confirmation of whether the conveyor belt is still in an abnormal motion state is achieved, thus avoiding false alarms of conveyor belt deviation and frequent alarms, thereby improving the efficiency of the abnormal motion detection method.

[0117] See Figure 9-10 , Figure 9 This is a flow chart of the seventh embodiment of the abnormal motion state detection method provided by this application. Figure 10 This is a schematic diagram of the detection interface of another embodiment of the terminal device provided by this application. Figure 9 As shown, the abnormal motion state detection method also includes:

[0118] S601: Acquire an image to be detected and a preset area 10 on the image to be detected.

[0119] When performing conveyor belt abnormality detection, the terminal device first obtains the image to be detected and divides the image to be detected into a preset area 10. The preset area 10 is roughly symmetrical about the conveyor belt, and the preset area 10 should try to include the conveyor belt area 20 and all roller detection frames 30.

[0120] S602 : generating a circumscribed rectangular area 40 based on the preset area 10 .

[0121] The terminal device determines the circumscribed rectangular area 40 of the preset area 10 based on the preset area 10. Figure 2 As shown, in one embodiment, if the conveyor belt is within the preset area 10 and does not run out of the roller, at this time, a minimum circumscribed rectangular area 40a of the preset area 10 can be directly generated based on the bottom edge of the preset area 10, and the minimum circumscribed rectangular area 40a can be used as the circumscribed rectangular area 40.

[0122] like Figure 10 As shown, in another embodiment, if part of the conveyor belt 20a extends beyond the rollers, that is, part of the conveyor belt 20a is outside the predetermined area 10, then the minimum circumscribed rectangular area 40a of the predetermined area 10 can be expanded outward by a number of pixels to generate a circumscribed rectangular area 40. The circumscribed rectangular area 40 includes at least part of the conveyor belt 20a to ensure that the entire conveyor belt area 20 is within the field of view of the circumscribed rectangular area 40 when the conveyor belt runs away.

[0123] S603: cropping the image to be inspected according to the circumscribed rectangular area 40, and identifying the conveyor belt area 20 and several target device areas of the image to be inspected in the cropped image to be inspected.

[0124] After generating the bounding rectangle 40, the terminal device crops the image to be inspected according to the bounding rectangle 40 to reduce the time it takes to process the image. The terminal device then identifies the conveyor belt area 20 and several target device areas within the cropped image. The target device areas include the roller detection frame 30.

[0125] Specifically, the terminal device is equipped with a first model and a second model. The first model is used to identify the conveyor belt area 20 in the image to be detected, and the second model is used to identify the roller detection frames 30 in the image to be detected. The first model is a semantic segmentation model, and the second model is an object detection model. Because the conveyor belt has a relatively regular shape, and the rollers are easily obscured by the conveyor belt when the conveyor belt deflects, it is difficult to form a regular pattern. In this embodiment, the semantic segmentation model is used to identify the conveyor belt area 20 in the image to be detected, and the object detection model is used to identify the roller detection frames 30 in the image to be detected.

[0126] The terminal device scales the cropped image to be detected and inputs it into the trained first model. The first model segments the conveyor belt of the image to be detected according to the set semantic label to form a conveyor belt area 20. The terminal device scales the cropped image to be detected and inputs it into the trained second model. The second model extracts the main features of the roller and predicts them, and outputs several roller detection frames 30.

[0127] After the terminal device generates the circumscribed rectangular area 40, the terminal device crops the image to be detected according to the circumscribed rectangular area 40, scales the cropped image to be detected to the first image size and inputs it into the trained first model. The first model segments the conveyor belt of the image to be detected and identifies the conveyor belt area 20 of the image to be detected. At the same time, the terminal device scales the cropped image to be detected to the second image size and inputs it into the trained second model. The second model extracts the main features of the roller and predicts them, identifying several roller detection frames 30. Optionally, the first model can be a DlinkNet network model or other semantic segmentation model; the second model can be a Yolov4 model or other target detection model; the first image size and the second image size are determined based on the selected model type. No specific limitation is made here on the first model and the second model.

[0128] In an embodiment of the present application, a terminal device acquires an image to be detected and a preset area 10 on the image to be detected; generates a circumscribed rectangular area 40 based on the preset area 10; crops the image to be detected according to the circumscribed rectangular area 40, and identifies a conveyor belt area 20 and several target device areas in the cropped image to be detected. The abnormal motion state detection method of this embodiment can effectively reduce the time it takes for the terminal device to process the image to be detected by cropping the image to be detected into the circumscribed rectangular area 40. After cropping, a semantic segmentation model is used to identify the conveyor belt area 20 in the image to be detected, and a target detection model is used to identify several roller detection frames 30 in the image to be detected, effectively improving the efficiency of conveyor belt anomaly detection.

[0129] See Figure 11 , Figure 11 FIG2 is a schematic diagram of a terminal device according to an embodiment of the present invention. The terminal device includes a memory 52 and a processor 51 connected to each other.

[0130] The memory 52 is used to store program instructions for implementing the abnormal motion state detection method described in any one of the above embodiments.

[0131] The processor 51 is configured to execute program instructions stored in the memory 52 .

[0132] The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signaling processing capabilities. The processor 51 may also be a general-purpose processor, a digital signaling processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0133] The memory 52 can be a memory stick, a TF card, etc., which can store all the information in the terminal device, including the input raw data, computer programs, intermediate operation results and final operation results. It is stored in the memory. It stores and retrieves information according to the location specified by the controller. With the memory, the string matching prediction device has a memory function and can ensure normal operation. The memory of the string matching prediction device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its purpose. There is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0135] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, system server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application.

[0138] See also Figure 12 , Figure 12 : It is a structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium of the present application stores a program file 61 that can implement all the above-mentioned abnormal motion state detection methods, wherein the program file 61 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the various implementation methods of the present application. The aforementioned storage device includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a target detection device such as a computer, a server, a mobile phone, or a tablet.

[0139] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal motion state of a conveyor belt, characterized in that: The method comprises: Segmenting a conveyor belt area and various target device areas from the image to be detected; the conveyor belt area includes the conveyor belt, and the target device area includes devices for transporting the conveyor belt; determining whether the conveyor belt is in an abnormal motion state based on a distribution relationship between the conveyor belt area and each of the target device areas; The determining whether the conveyor belt is in an abnormal motion state based on the distribution relationship between the conveyor belt area and each of the target device areas includes: Determining a comprehensive deviation parameter based on the distribution relationship and the area of ​​each target device region; determining whether the conveyor belt is in an abnormal motion state according to the magnitude of the deviation comprehensive parameter and the deviation threshold; Determining the comprehensive deviation parameter based on the distribution relationship and the area of ​​each target device region includes: Determining a first area of ​​each of the target device regions distributed on a first side of the conveyor belt region; and determining a second area of ​​each of the target device regions distributed on a second side of the conveyor belt region; Determine the area of ​​the background region, where the area of ​​the background region is the area of ​​the region excluding the conveyor belt region and each of the target device regions in the image to be detected; based on the first area, the second area, the background area, and the determined total number of target device areas; A comprehensive deviation parameter is determined based on the background area, the total number of target device areas, the first area and / or the second area.

2. The abnormal motion state detection method according to claim 1, characterized in that: Determining the comprehensive deviation parameter based on the distribution relationship and the area of ​​each target device region includes: Determining a first area of ​​each of the target device regions distributed on a first side of the conveyor belt region; and determining a second area of ​​each of the target device regions distributed on a second side of the conveyor belt region; The comprehensive deviation parameter is determined based on the first area and the second area.

3. The abnormal motion state detection method according to claim 2, characterized in that: The determining the comprehensive deviation parameter based on the first area and the second area includes: Determine the area of ​​the background region, where the area of ​​the background region is the area of ​​the region excluding the conveyor belt region and each of the target device regions in the image to be detected; The comprehensive deviation parameter is determined based on the first area, the second area, and the background area.

4. The abnormal motion state detection method according to claim 3, characterized in that: The determining the comprehensive deviation parameter based on the first area, the second area, and the background area includes: The deviation comprehensive parameter is determined based on the first area, the second area, the background area, and the determined total number of target device areas.

5. The abnormal motion state detection method according to any one of claims 1 to 4, characterized in that: The determining whether the conveyor belt is in an abnormal motion state according to the magnitude of the deviation comprehensive parameter and the deviation threshold includes: In response to the comprehensive deviation parameter being greater than or equal to the deviation threshold, it is determined that the conveyor belt is in an abnormal movement state.

6. The abnormal motion state detection method according to claim 5, characterized in that: The abnormal motion state includes multiple deviation states of different levels; After determining that the conveyor belt is in an abnormal motion state in response to the comprehensive deviation parameter being greater than or equal to the deviation threshold, the method further includes: Determining a comprehensive deviation parameter range of the comprehensive deviation parameter mapping based on comprehensive deviation parameter ranges mapped to different deviation states among the multiple deviation states; The deviation state mapped by the determined deviation comprehensive parameter range is determined as the motion state of the conveyor belt.

7. The abnormal motion state detection method according to claim 1, characterized in that: After determining whether the conveyor belt is in an abnormal motion state, the abnormal motion state detection method further includes: Pre-set alarm time interval; When it is determined that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time begins to be accumulated; During the duration of confirming that the conveyor belt is in an abnormal motion state, when the accumulated alarm time reaches the alarm time interval, an alarm signal is issued and the alarm time is reset.

8. The abnormal motion state detection method according to claim 7, characterized in that: The alarm time interval includes a pre-alarm time interval and a repeated alarm time interval; the abnormal motion state detection method further includes: When it is confirmed that the conveyor belt is in an abnormal motion state, the alarm time is initialized and the alarm time begins to be accumulated; When the accumulated duration of the alarm time reaches the pre-alarm time interval, determining whether the conveyor belt is in an abnormal motion state; If yes, continue to accumulate the alarm time until the repeated alarm time interval is reached, and issue an alarm signal; If not, reset the alarm time.

9. The abnormal motion state detection method according to claim 1, characterized in that: The abnormal motion state detection method further includes: Acquire the image to be detected and a preset area on the image to be detected; Generate a circumscribed rectangular area based on the preset area; The image to be detected is cropped according to the circumscribed rectangular area, and the conveyor belt area and the target device areas of the image to be detected are segmented from the cropped image to be detected.

10. A terminal device, characterized in that: The terminal device includes a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the abnormal motion state detection method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and when the program instructions are executed, the abnormal motion state detection method according to any one of claims 1 to 9 is implemented.

Images