Method, processor, apparatus and storage medium for detecting blender failure

Abstract

The application relates to the field of engineering machinery, in particular to a method for detecting a mixer fault, a processor, an apparatus and a storage medium. The method comprises the following steps: acquiring a target area image inside a mixer host; determining a current pose of a target stirring blade contained in the target area image and a target point cloud corresponding to the target area image according to the target area image; acquiring an initial mechanical model inside the mixer host; determining an initial pose of the target stirring blade in the initial mechanical model and an initial point cloud corresponding to the initial mechanical model according to the initial mechanical model; rotating and iterating the initial point cloud until the initial pose is consistent with the current pose, and then determining an actual point cloud inside the mixer host; comparing the actual point cloud with the target point cloud, and determining an offset distance between the target point cloud and the actual point cloud; and determining whether a target area of the mixer has a fault according to the offset distance. The above scheme can timely find the fault of the mixer.

Description

Technical Field

[0001] This application relates to the field of construction machinery, and more specifically, to a method, processor, apparatus, and storage medium for detecting mixer malfunctions. Background Technology

[0002] In the concrete production process, the mixer operates in a closed system, and various malfunctions can occur during operation. These include shaft seizure (concrete lumps forming on the mixing shaft or blades) and wear-related issues (wear on the mixing shaft). Currently, due to the closed nature of the mixer, it's difficult to directly understand the internal condition of the main unit. Current methods for troubleshooting concrete mixer malfunctions primarily involve long-term monitoring and statistical analysis of malfunction occurrence cycles. These cycles are then used to establish manual inspection schedules for periodic checks of the main mixer's internal components. However, this periodic approach can lead to delayed troubleshooting. Furthermore, this method can be indiscriminate in its search for faults, as it involves inspecting components that are not malfunctioning, wasting manpower and impacting production efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a method, processor, device, and storage medium for automatically detecting mixer malfunctions.

[0004] To achieve the above objectives, this application provides a method for detecting mixer malfunctions. The mixer includes a mixing shaft and multiple mixing blades. The method includes:

[0005] Acquire an image of the target area inside the mixer main unit;

[0006] The current pose of the target stirring blade contained in the target region image and the target point cloud corresponding to the target region image are determined based on the target region image.

[0007] Obtain the initial mechanical model of the mixer's main unit;

[0008] The initial pose of the target stirring blade within the initial mechanical model and the initial point cloud corresponding to the initial mechanical model are determined based on the initial mechanical model.

[0009] The initial point cloud is rotated and iterated until the initial pose matches the current pose, thus determining the actual point cloud inside the mixer host.

[0010] Compare the actual point cloud with the target point cloud and determine the offset distance between the target point cloud and the actual point cloud;

[0011] The offset distance is used to determine whether a malfunction has occurred in the target area of ​​the mixer.

[0012] In this embodiment of the application, determining whether a target area of ​​the mixer has malfunctioned based on the offset distance includes: determining that the target area has a wear fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation towards the center of the mixing shaft reaches a preset wear threshold; and determining that the target area has a shaft-clamping fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation away from the center of the mixing shaft reaches a first preset shaft-clamping threshold.

[0013] In this embodiment of the application, the method further includes: when the offset distance is greater than a first preset bearing seizure threshold and less than or equal to a second preset bearing seizure threshold, determining the bearing seizure fault as mild bearing seizure; when the offset distance is greater than a second preset bearing seizure threshold and less than or equal to a third preset bearing seizure threshold, determining the bearing seizure fault as moderate bearing seizure; and when the offset distance is greater than a third preset bearing seizure threshold, determining the bearing seizure fault as severe bearing seizure.

[0014] In this embodiment of the application, the target point cloud includes multiple target points. Comparing the actual point cloud with the target point cloud and determining the offset distance between the target point cloud and the actual point cloud includes: determining the shortest spatial distance between each target point in the target point cloud and the actual point cloud; and determining each shortest spatial distance as the offset distance between the target point cloud and the actual point cloud.

[0015] In this embodiment of the application, the mixer further includes an image acquisition device. Acquiring an image of the target area inside the mixer main unit includes: acquiring a target image of the inside of the mixer main unit through the image acquisition device; determining the area where the stirring shaft and stirring blades are located in the target image as the region of interest; and cropping the target image according to the region of interest to obtain the target area image.

[0016] In this embodiment of the application, determining the target point cloud corresponding to the target region image includes: determining the image point cloud corresponding to the target region image; obtaining the mask of the region where the stirring shaft and stirring blade are located in the target region image by Gaussian background modeling; and determining the target point cloud corresponding to the target region image based on the mask and the image point cloud, wherein the target point cloud is the point cloud corresponding to the stirring shaft and stirring blade contained in the target region image.

[0017] In this embodiment of the application, the process of rotating and iterating the initial point cloud until the initial pose matches the current pose, and then determining the actual point cloud inside the mixer host, includes: rotating and iterating the initial point cloud according to the rotation direction of the stirring shaft until the initial pose matches the current pose, and then determining the rotation matrix between the initial pose and the current pose; transforming the initial point cloud through the rotation matrix to obtain the actual point cloud inside the mixer host.

[0018] A second aspect of this application provides a processor configured to perform any of the above-described methods for detecting mixer malfunctions.

[0019] A third aspect of this application provides a mixer, the mixer comprising:

[0020] Multiple stirring blades are used to stir the materials inside the mixer;

[0021] A stirring shaft is used to mount multiple stirring blades and control the rotation of these blades.

[0022] An image detection device for acquiring images of the interior of the mixer main unit; and the aforementioned processor.

[0023] A fourth aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method for detecting a mixer malfunction according to any one of the preceding claims.

[0024] The above technical solution involves the processor acquiring images of the internal workings of the mixer via an image acquisition device. By comparing the target point cloud of the target area acquired by the image acquisition device with the actual point cloud obtained based on the mixer's initial mechanical model and current pose, the offset distance between the target point cloud and the actual point cloud is determined. Based on this offset distance, it is determined whether a mixer malfunction has occurred. This eliminates the need for manual inspection of the mixer, allowing for timely detection of malfunctions and notification of maintenance personnel, ensuring the mixer's normal operation and improving production quality and efficiency.

[0025] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0026] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the following detailed description to explain the present application, but do not constitute a limitation thereof. In the drawings:

[0027] Figure 1 A flowchart illustrating a method for detecting a mixer malfunction according to an embodiment of this application is shown schematically.

[0028] Figure 2 The schematic diagram illustrates the structure of a mixer according to an embodiment of this application;

[0029] Figure 3 The diagram illustrates the internal structure of a computer device according to an embodiment of this application. Detailed Implementation

[0030] The specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0031] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0032] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0033] like Figure 1 The diagram illustrates a flowchart of a method for detecting a mixer malfunction according to an embodiment of this application. Figure 1 As shown, a method for detecting mixer malfunctions is provided, comprising the following steps:

[0034] Step 101: Obtain an image of the target area inside the mixer main unit;

[0035] Step 102: Determine the current pose of the target stirring blade contained in the target region image and the target point cloud corresponding to the target region image based on the target region image;

[0036] Step 103: Obtain the initial mechanical model of the inside of the mixer main unit;

[0037] Step 104: Determine the initial pose of the target stirring blade within the initial mechanical model and the initial point cloud corresponding to the initial mechanical model based on the initial mechanical model.

[0038] Step 105: Rotate and iterate the initial point cloud until the initial pose matches the current pose, then determine the actual point cloud inside the mixer host.

[0039] Step 106: Compare the actual point cloud with the target point cloud and determine the offset distance between the target point cloud and the actual point cloud;

[0040] Step 107: Determine whether a fault has occurred in the target area of ​​the mixer based on the offset distance.

[0041] The mixer may include a stirring shaft and multiple stirring blades. A processor can acquire an image of a target area inside the mixer's main unit via an image acquisition device. The processor can process this image, determine the stirring blades contained within the target area image, identify these blades as target stirring blades, and determine their current pose. The processor can also acquire the target point cloud corresponding to the target area image. Since the image acquisition device captures a two-dimensional image, the target point cloud corresponding to the target area image only contains partial information about the stirring shaft and stirring blades; that is, the point cloud corresponding to the information of a surface captured by the image acquisition device.

[0042] The processor can acquire the initial mechanical model of the mixer's internal structure. The information of the initial mechanical model can be provided and stored by the manufacturer when the mixer leaves the factory. Since the mixer includes multiple mixing blades, the initial mechanical model includes multiple mixing blades. The processor can identify the target mixing blade in the initial mechanical model that is the same as the mixing blade contained in the target region image, and determine the initial pose of the target mixing blade within the initial mechanical model. The processor can also acquire the initial point cloud corresponding to the initial mechanical model.

[0043] The processor can perform rotational iterative processing on the initial point cloud corresponding to the initial mechanical model until the initial pose of the target stirring blade in the initial mechanical model matches the current pose of the target stirring blade in the target region image. The processor can determine the rotation angle between the initial pose and the current pose, and process the initial point cloud based on the obtained rotation angle to obtain the actual point cloud inside the mixer main unit. That is, the point cloud corresponding to the initial mechanical model after rotating to the same pose as the mixer in the target region image. The processor can compare the actual point cloud inside the mixer main unit with the target point cloud in the target region image and determine the offset distance between the target point cloud and the actual point cloud. The processor can determine whether a fault has occurred in the target area of ​​the mixer based on the determined offset distance.

[0044] In one embodiment, determining whether a target area of ​​the mixer has malfunctioned based on the offset distance includes: determining that the target area has a wear fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation towards the center of the mixing shaft reaches a preset wear threshold; and determining that the target area has a shaft-clamping fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation away from the center of the mixing shaft reaches a first preset shaft-clamping threshold.

[0045] The processor compares the actual point cloud inside the mixer main unit with the target point cloud of the target area image. When the processor determines that the target point cloud is closer to the center rotation axis of the mixing shaft relative to the actual point cloud, the processor can determine that the mixer in the target area has experienced wear. When the processor determines that the offset distance of the target point cloud relative to the actual point cloud towards the center rotation axis of the mixing shaft reaches a preset wear threshold set by the processor, the processor can determine that a wear fault has occurred in the target area. The preset wear threshold can be determined by summarizing and analyzing historical data of the mixer or by user input. When the processor determines that the target point cloud is farther away from the center rotation axis of the mixing shaft relative to the actual point cloud, the processor can determine that the mixer in the target area has experienced shaft seizure, that is, concrete clumps have appeared on the mixing shaft or mixing blades. When the processor determines that the offset distance of the target point cloud relative to the actual point cloud away from the center rotation axis of the mixing shaft reaches a first preset shaft seizure threshold set by the processor, the processor can determine that a shaft seizure fault has occurred in the target area.

[0046] In one embodiment, the method further includes: determining the bearing failure as a mild bearing failure when the offset distance is greater than a first preset bearing failure threshold and less than or equal to a second preset bearing failure threshold; determining the bearing failure as a moderate bearing failure when the offset distance is greater than the second preset bearing failure threshold and less than or equal to a third preset bearing failure threshold; and determining the bearing failure as a severe bearing failure when the offset distance is greater than the third preset bearing failure threshold.

[0047] When the processor determines that the position of the target point cloud is further away from the center rotation axis of the mixing shaft relative to the actual point cloud, the processor can determine that the mixer in the target area has a shaft seizure fault. The processor can further detect the offset distance. When the offset distance is greater than the first preset shaft seizure threshold set by the processor and less than or equal to the second preset shaft seizure threshold set by the processor, the processor can determine that the current mixer has a slight shaft seizure. When the offset distance is greater than the second preset shaft seizure threshold set by the processor and less than or equal to the third preset shaft seizure threshold set by the processor, the processor can determine that the shaft seizure fault is moderate. When the offset distance is greater than the third preset shaft seizure threshold set by the processor, the processor determines that the shaft seizure fault is severe. The first, second, and third preset shaft seizure thresholds can be determined by the processor through analysis of historical shaft seizure cases or by user input. After the processor determines different degrees of shaft seizure within the mixer based on the offset distance, the user can perform corresponding repairs based on the degree of shaft seizure determined by the processor.

[0048] In one embodiment, the target point cloud includes multiple target points. Comparing the actual point cloud with the target point cloud and determining the offset distance between the target point cloud and the actual point cloud includes: determining the shortest spatial distance between each target point in the target point cloud and the actual point cloud; and determining each shortest spatial distance as the offset distance between the target point cloud and the actual point cloud.

[0049] After obtaining the actual point cloud inside the mixer, the processor can compare the actual point cloud with the target point cloud corresponding to the target region image. The actual point cloud is the point cloud corresponding to the initial mechanical model in the same pose as the mixer in the target region image. The target point cloud is the point cloud corresponding to the target region image, containing multiple target points. The processor can determine the shortest spatial distance between each target point in the target point cloud and the actual point cloud, and define this shortest spatial distance as the offset distance between the target point and the actual point cloud. After determining the shortest spatial distance between each target point in the target point cloud and the actual point cloud, the processor can define each shortest spatial distance as the offset distance between the target point cloud and the actual point cloud, and judge each offset distance to determine whether a fault has occurred in the target area of ​​the mixer, and the type of fault. For example, the processor can detect all offset distances between the target point cloud and the actual point cloud, and determine all target points corresponding to offset distances greater than a preset wear threshold set by the processor as areas in the target region where wear faults have occurred.

[0050] In one embodiment, the mixer further includes an image acquisition device, and acquiring an image of a target area inside the mixer main unit includes: acquiring a target image of the inside of the mixer main unit through the image acquisition device; determining the area where the stirring shaft and stirring blades are located in the target image as the region of interest; and cropping the target image according to the region of interest to obtain the target area image.

[0051] The mixer includes a mixing shaft and multiple mixing blades for mixing concrete. An image acquisition device can capture images of the interior of the mixer to obtain a target image. After the processor acquires the target image, it can divide the area containing the mixing shaft and blades into regions of interest (ROIs). The target image is then cropped based on these ROIs to obtain the target region image.

[0052] In one embodiment, determining the target point cloud corresponding to the target region image includes: determining the image point cloud corresponding to the target region image; obtaining a mask of the region where the stirring shaft and stirring blade are located in the target region image by Gaussian background modeling; and determining the target point cloud corresponding to the target region image based on the mask and the image point cloud, wherein the target point cloud is the point cloud corresponding to the stirring shaft and stirring blade contained in the target region image.

[0053] After the processor acquires the target area image inside the mixer host, it can determine the image point cloud of the target area image. By modeling the background using Gaussian mixture, it obtains the mask of the area where the stirring shaft and stirring blades are located in the target area image. The processor can extract the image point cloud based on the mask, thereby obtaining the point cloud corresponding to the stirring shaft and stirring blades in the target area image, and determining the point cloud corresponding to the stirring shaft and stirring blades in the target area image as the target point cloud corresponding to the target area image.

[0054] In one embodiment, determining the actual point cloud inside the mixer host by rotating and iterating the initial point cloud until the initial pose matches the current pose includes: rotating and iterating the initial point cloud along the rotation direction of the mixing shaft until the initial pose matches the current pose, then determining the rotation matrix between the initial pose and the current pose; and transforming the initial point cloud using the rotation matrix to obtain the actual point cloud inside the mixer host.

[0055] After determining the initial point cloud of the mixer based on the initial mechanical model inside the mixer host, the processor can identify the stirring blade in the initial mechanical model that matches the one in the target region image, and designate it as the target stirring blade. For example, assuming the stirring blade in the target region image is the second stirring blade on the stirring shaft of the mixer, the processor can identify the second stirring blade in the initial mechanical model as the target stirring blade. The processor can determine the initial pose of the target stirring blade and iterate the initial point cloud according to the rotation direction of the stirring shaft until the initial pose of the target stirring blade matches the current pose of the target stirring blade in the target region image. Then, the processor can determine the rotation matrix between the initial pose and the current pose. The processor then transforms the initial point cloud using the rotation matrix to obtain the actual point cloud inside the mixer host, which is the point cloud corresponding to the initial mechanical model being in the same pose as the mixer in the target region image.

[0056] In one embodiment, such as Figure 2 The diagram illustrates a schematic structural diagram of a mixer 200, which includes: a plurality of stirring blades 201 for stirring materials within the mixer 200; a stirring shaft 202 for mounting the plurality of stirring blades 201 and controlling the rotation of the plurality of stirring blades 201; an image detection device 203 for acquiring images of the interior of the mixer 200; and a processor 204 for performing a method for detecting mixer malfunctions that executes any of the above.

[0057] In one embodiment, a processor is provided, configured to perform any of the foregoing methods for detecting a mixer malfunction.

[0058] The processor can acquire a target image of the interior of the mixer main unit through an image detection device. After acquiring the target image, the processor can divide the area containing the stirring shaft and stirring blades into regions of interest (ROIs). The processor then crops the target image based on the defined ROIs to obtain the target region image. After acquiring the target region image of the interior of the mixer main unit, the processor can determine the image point cloud of the target region image. Through Gaussian background mixing modeling, it obtains a mask for the area containing the stirring shaft and stirring blades in the target region image. The processor can then extract the image point cloud based on the mask, thereby obtaining the point cloud corresponding to the stirring shaft and stirring blades in the target region image, and defining the point cloud corresponding to the stirring shaft and stirring blades in the target region image as the target point cloud corresponding to the target region image.

[0059] The processor can also directly acquire point cloud information of the mixer's interior using a stereo camera (e.g., RealSense D435). It then divides the area containing the mixing shaft and blades into target regions, determining the point cloud information within these regions. The processor further uses Gaussian mixture background modeling to obtain a mask for the mixing shaft and blades in the target region image. Based on this mask, the processor extracts the point cloud information, obtaining the point cloud corresponding to the mixing shaft and blades in the target region image, and identifies this point cloud as the target point cloud. The processor can also acquire the initial mechanical model of the mixer's interior, which can be provided and stored by the manufacturer at the time of manufacture. The processor then determines the initial point cloud of the mixer based on this initial mechanical model.

[0060] The processor can determine the corresponding target mixing blade from the target region image or target point cloud. This means identifying the mixing blade contained in the target region image or the mixing blade corresponding to the target point cloud. It then identifies the mixing blade in the initial mechanical model that matches the target mixing blade, thus designating it as the target mixing blade of the initial mechanical model. The processor can determine the initial pose of the target mixing blade in the initial mechanical model and iteratively rotate the initial point cloud along the rotation direction of the mixing shaft until the initial pose of the target mixing blade in the initial mechanical model matches the current pose of the target mixing blade in the target region image. Then, it can determine the rotation matrix between the initial pose of the target mixing blade in the initial mechanical model and the current pose of the target mixing blade in the target region image. The processor then transforms the initial point cloud using the rotation matrix to obtain the actual point cloud inside the mixer, which is the point cloud corresponding to the initial mechanical model when it is in the same pose as the mixer in the target region image. For example, assuming the mixing blade in the target region image is the second mixing blade on the mixing shaft of the mixer, the processor can identify the second mixing blade in the initial mechanical model as the target mixing blade. The processor can determine the current pose of the second stirring blade in the target region image and the initial pose of the second stirring blade in the initial mechanical model. The processor iterates the rotation of the initial point cloud corresponding to the initial mechanical model in the direction of the stirring shaft until the initial pose of the second stirring blade in the initial mechanical model is consistent with the current pose of the second stirring blade in the target region image. Then the processor can determine the rotation matrix between the current pose and the initial pose and transform the initial point cloud through the rotation matrix to obtain the actual point cloud.

[0061] After obtaining the actual point cloud inside the mixer, the processor can compare the actual point cloud with the target point cloud corresponding to the target region image. The actual point cloud is the point cloud corresponding to the initial mechanical model in the same pose as the mixer in the target region image, while the target point cloud is the point cloud corresponding to the target region image. The target point cloud includes multiple target points. The processor can determine the shortest spatial distance between each target point in the target point cloud and the actual point cloud, and define this shortest spatial distance as the offset distance between the target point and the actual point cloud. After determining the shortest spatial distance between each target point in the target point cloud and the actual point cloud, the processor can define each shortest spatial distance as the offset distance between the target point cloud and the actual point cloud, and judge each offset distance to determine whether a fault has occurred in the target area of ​​the mixer, and the type of fault.

[0062] When the processor determines that the position of the target point cloud is closer to the center rotation axis of the mixing shaft relative to the actual point cloud, the processor can determine that the mixer in the target area has experienced wear. When the processor determines that the offset distance of the target point cloud relative to the actual point cloud towards the center rotation axis of the mixing shaft reaches a preset wear threshold set by the processor, the processor can determine that a wear fault has occurred in the target area. The preset wear threshold can be determined by summarizing and analyzing historical data of the mixer, or it can be determined by user input. When the processor determines that the position of the target point cloud is farther away from the center rotation axis of the mixing shaft relative to the actual point cloud, the processor can determine that the mixer in the target area has experienced shaft seizure, that is, concrete clumps have appeared on the mixing shaft or mixing blades. When the processor determines that the offset distance of the target point cloud relative to the actual point cloud away from the center rotation axis of the mixing shaft reaches a first preset shaft seizure threshold set by the processor, the processor can determine that a shaft seizure fault has occurred in the target area.

[0063] The above technical solution involves the processor acquiring images of the internal workings of the mixer via an image acquisition device. By comparing the target point cloud of the target area acquired by the image acquisition device with the actual point cloud obtained based on the mixer's initial mechanical model and current pose, the offset distance between the target point cloud and the actual point cloud is determined. Based on this offset distance, it is determined whether the mixer has malfunctioned. If a malfunction is confirmed, the type of malfunction can be further determined using the offset distance. This eliminates the need for manual inspection of the mixer. It allows for timely detection and handling of mixer malfunctions, ensuring the mixer's normal operation and thus improving production quality and efficiency.

[0064] In one embodiment, a machine-readable storage medium is provided, on which instructions are stored, which, when executed by a processor, cause the processor to be configured to perform a method for detecting a mixer malfunction according to any one of the foregoing.

[0065] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0066] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown. The computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database stores relevant data about the construction machinery and data input by the operators. The network interface A02 communicates with external terminals via a network connection. When executed by the processor A01, the computer program B02 implements a method for detecting mixer malfunctions.

[0067] Figure 1 This is a flowchart illustrating a method for detecting a mixer malfunction in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0068] This application provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring an image of a target area inside a mixer host; determining the current pose of a target stirring blade contained in the target area image and a target point cloud corresponding to the target area image based on the target area image; acquiring an initial mechanical model inside the mixer host; determining the initial pose of the target stirring blade within the initial mechanical model and an initial point cloud corresponding to the initial mechanical model based on the initial mechanical model; rotating and iterating the initial point cloud until the initial pose matches the current pose, then determining the actual point cloud inside the mixer host; comparing the actual point cloud with the target point cloud and determining the offset distance between the target point cloud and the actual point cloud; and determining whether a fault has occurred in the target area of ​​the mixer based on the offset distance.

[0069] In one embodiment, determining whether a target area of ​​the mixer has malfunctioned based on the offset distance includes: determining that the target area has a wear fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation towards the center of the mixing shaft reaches a preset wear threshold; and determining that the target area has a shaft-clamping fault when the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation away from the center of the mixing shaft reaches a first preset shaft-clamping threshold.

[0070] In one embodiment, the method further includes: determining the bearing failure as a mild bearing failure when the offset distance is greater than a first preset bearing failure threshold and less than or equal to a second preset bearing failure threshold; determining the bearing failure as a moderate bearing failure when the offset distance is greater than the second preset bearing failure threshold and less than or equal to a third preset bearing failure threshold; and determining the bearing failure as a severe bearing failure when the offset distance is greater than the third preset bearing failure threshold.

[0071] In one embodiment, the target point cloud includes multiple target points. Comparing the actual point cloud with the target point cloud and determining the offset distance between the target point cloud and the actual point cloud includes: determining the shortest spatial distance between each target point in the target point cloud and the actual point cloud; and determining each shortest spatial distance as the offset distance between the target point cloud and the actual point cloud.

[0072] In one embodiment, the mixer further includes an image acquisition device, and acquiring an image of a target area inside the mixer main unit includes: acquiring a target image of the inside of the mixer main unit through the image acquisition device; determining the area where the stirring shaft and stirring blades are located in the target image as the region of interest; and cropping the target image according to the region of interest to obtain the target area image.

[0073] In one embodiment, determining the target point cloud corresponding to the target region image includes: determining the image point cloud corresponding to the target region image; obtaining a mask of the region where the stirring shaft and stirring blade are located in the target region image by Gaussian background modeling; and determining the target point cloud corresponding to the target region image based on the mask and the image point cloud, wherein the target point cloud is the point cloud corresponding to the stirring shaft and stirring blade contained in the target region image.

[0074] In one embodiment, determining the actual point cloud inside the mixer host by rotating and iterating the initial point cloud until the initial pose matches the current pose includes: rotating and iterating the initial point cloud along the rotation direction of the mixing shaft until the initial pose matches the current pose, then determining the rotation matrix between the initial pose and the current pose; and transforming the initial point cloud using the rotation matrix to obtain the actual point cloud inside the mixer host.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0081] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0082] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0083] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting mixer malfunctions, characterized in that, The mixer includes a stirring shaft and multiple stirring blades, and the method includes: Obtain an image of the target area inside the mixer main unit; The current pose of the target stirring blade contained in the target region image and the target point cloud corresponding to the target region image are determined based on the target region image. Obtain the initial mechanical model of the interior of the mixer main unit; The initial pose of the target stirring blade within the initial mechanical model and the initial point cloud corresponding to the initial mechanical model are determined based on the initial mechanical model. The initial point cloud is rotated and iterated until the initial pose matches the current pose, then the actual point cloud inside the mixer host is determined. Compare the actual point cloud with the target point cloud, and determine the offset distance between the target point cloud and the actual point cloud; The target area of ​​the mixer is determined based on the offset distance to determine whether a malfunction has occurred.

2. The method for detecting mixer malfunctions according to claim 1, characterized in that, The step of determining whether a fault has occurred in the target area of ​​the mixer based on the offset distance includes: If the offset distance of the target point cloud relative to the actual point cloud in the direction of rotation towards the center of the stirring shaft reaches a preset wear threshold, it is determined that the target area has a wear fault. If the offset distance of the target point cloud relative to the actual point cloud in the direction away from the central rotation axis of the stirring shaft reaches a first preset shaft-clamping threshold, it is determined that a shaft-clamping fault has occurred in the target area.

3. The method for detecting mixer malfunctions according to claim 2, characterized in that, The method further includes: When the offset distance is greater than the first preset bearing threshold and less than or equal to the second preset bearing threshold, the bearing failure is determined to be a mild bearing failure. When the offset distance is greater than the second preset bearing clamping threshold and less than or equal to the third preset bearing clamping threshold, the bearing clamping fault is determined to be a moderate bearing clamping fault. If the offset distance is greater than the third preset bearing seizure threshold, the bearing seizure fault is determined to be severe bearing seizure.

4. The method for detecting mixer malfunctions according to claim 1, characterized in that, The target point cloud includes multiple target points, and the step of comparing the actual point cloud with the target point cloud and determining the offset distance between the target point cloud and the actual point cloud includes: Determine the shortest spatial distance between each target point in the target point cloud and the actual point cloud; Each shortest spatial distance is defined as the offset distance between the target point cloud and the actual point cloud.

5. The method for detecting mixer malfunctions according to claim 1, characterized in that, The mixer also includes an image acquisition device, wherein acquiring an image of the target area inside the mixer main unit includes: The target image inside the mixer main unit is acquired through the image acquisition device; The region in the target image containing the stirring shaft and the stirring blades is identified as the region of interest. The target image is cropped according to the region of interest to obtain the target region image.

6. The method for detecting mixer malfunctions according to claim 1, characterized in that, Determining the target point cloud corresponding to the target region image includes: Determine the image point cloud corresponding to the target region image; By modeling the background using Gaussian mixture, the mask of the region where the stirring axis and the stirring blade are located in the target region image is obtained; The target point cloud corresponding to the target region image is determined based on the mask and the image point cloud, wherein the target point cloud is the point cloud corresponding to the stirring shaft and stirring blade contained in the target region image.

7. The method for detecting mixer malfunctions according to claim 1, characterized in that, The step of rotating and iterating the initial point cloud until the initial pose matches the current pose, and then determining the actual point cloud inside the mixer host, includes: The initial point cloud is rotated iteratively along the rotation direction of the stirring shaft until the initial pose is consistent with the current pose, and then the rotation matrix between the initial pose and the current pose is determined. The initial point cloud is transformed using the rotation matrix to obtain the actual point cloud inside the mixer main unit.

8. A device for detecting mixer malfunctions, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting mixer malfunctions as described in any one of claims 1 to 7.

9. A mixer, characterized in that, The mixer includes: Multiple stirring blades are used to stir the materials inside the mixer; A stirring shaft is used to mount multiple stirring blades and control the rotation of the multiple stirring blades; Image detection device, used to acquire images of the interior of the mixer main unit; and Includes a processor that implements the method for detecting mixer malfunctions as described in any one of claims 1 to 7.

10. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the method for detecting a mixer malfunction according to any one of claims 1 to 7.

Images