Image-based belt detection method, device, equipment and medium

Abstract

The present disclosure relates to the field of image analysis technology, and provides an image-based belt detection method, device, equipment and medium, the method comprising: obtaining first detection data of a weight sensor, calculating the centroid of the first detection data; when the first detection data meets a preset condition, fusing the observation data of the first ultrasonic wave and the second ultrasonic wave, wherein the fusion comprises: converting K frames of first ultrasonic wave observation data into a depth image; determining the motion vector and difference information of key feature points between each two adjacent depth images, and extracting the first boundary contour based on the difference information; determining the gradient change of the second ultrasonic wave observation data along the directivity of the first boundary contour, and extracting the second boundary contour; calculating the ratio of the intersection area of ​​the first boundary contour and the second boundary contour to the union area, and when the ratio is greater than a threshold, taking the intersection area as the target area. By adopting the above scheme, the belt condition can be detected and analyzed with high precision and high reliability.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image analysis, and in particular to an image-based belt detection method, device, equipment and medium. Background Art

[0002] In industrial production, the transmission belt is a key equipment for material transportation, and its operation status is directly related to the stability and efficiency of the entire production line. However, due to factors such as external sharp foreign objects, the transmission belt may tear longitudinally during operation, which will not only cause material leakage, but also cause equipment damage, production interruption, and even cause safety accidents, bringing significant losses to the company.

[0003] In the prior art, there are a variety of belt detection and analysis methods based on traditional optical sensors, such as cameras, etc., but these methods often have some limitations in practical applications. First, although the traditional visual detection method is intuitive, it is greatly affected by factors such as ambient light, dust, and oil, and is prone to misjudgment and missed detection. Secondly, although the contact detection method is accurate, it requires the installation of sensors or probes on the belt. This method is only applicable to conductive belt materials and may also cause wear to the belt, affecting its service life. In addition, many detection methods cannot effectively identify cracks in a timely manner when the belt transport material is blocked.

[0004] Therefore, a method is needed to accurately and reliably perform belt detection and analysis. Summary of the invention

[0005] The present invention discloses an image-based belt detection method, device, equipment and medium for realizing high-precision and high-reliability belt detection and analysis.

[0006] In a first aspect, the present disclosure provides an image-based belt detection method, comprising:

[0007] Acquire first detection data of the weight sensor, and calculate the centroid of the first detection data;

[0008] When the first detection data meets the preset condition, the observation data of the first ultrasonic wave and the observation data of the second ultrasonic wave are fused, wherein the fusion includes:

[0009] Convert K first ultrasonic observation data into depth images and sort them according to observation timestamps, where K is a natural number greater than 1; determine motion vectors of key feature points between each two adjacent depth images, calculate difference information of K-1 motion vectors, extract a first boundary contour based on the difference information, and obtain the directivity of the first boundary contour, where the directivity refers to the longitudinal extension direction of the first boundary contour;

[0010] Taking the centroid of the first detection data as the starting point of the second ultrasonic observation data, determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour, identifying the boundary pixel points, and extracting the second boundary contour;

[0011] The ratio of the intersection area and the union area of ​​the first boundary contour and the second boundary contour is calculated, and when the ratio is greater than a threshold, the intersection area is taken as the target area.

[0012] According to the image-based belt detection method provided by the present disclosure, the motion vectors of the key feature points between each two adjacent depth images are determined, and the difference information of K-1 motion vectors is calculated, including:

[0013] Taking the centroid of the first detection data as a starting point, selecting image feature points whose weight information detected by the weight sensor within a preset distance reaches a first threshold as key feature points;

[0014] For each two adjacent depth images, match the key feature points in the current image with the corresponding key feature points in the previous image;

[0015] Calculate the angular change of the motion vector of the matched key feature point between each two adjacent depth images;

[0016] The difference information is calculated by the angular change of the motion vector.

[0017] According to the image-based belt detection method provided by the present disclosure, determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour includes:

[0018] The ultrasonic intensity variance value between each observation data and the observation data in the 8-neighborhood in the second ultrasonic observation data is calculated to obtain the gradient change.

[0019] According to the image-based belt detection method provided by the present disclosure, obtaining first detection data of the weight sensor includes:

[0020] The weight sensor is located below the initial position of the transmission belt in the form of an array unit;

[0021] The array unit is a weight sensor arranged in a two-dimensional array of M×N. The width of the array unit is equal to the width of the transmission belt. M and N are natural numbers.

[0022] The first detection data includes weight information and corresponding position information of each weight sensor.

[0023] According to the image-based belt detection method provided by the present disclosure, the first detection data meets the preset conditions including:

[0024] Traversing the sensor array: dividing the sensor array unit into multiple blocks, setting priorities for the multiple blocks respectively, wherein the priority of the central area of ​​the transmission belt is higher than that of other areas; using a multi-threaded method and traversing multiple blocks of the sensor array unit according to the priority level;

[0025] When the weight information detected by the traversed T adjacent weight sensors all reaches the first threshold, the centroids of the T adjacent weight sensors are obtained, and the X and Y coordinates of the centroids in the array unit are calculated.

[0026] According to the image-based belt detection method provided by the present disclosure, when the first detection data meets the preset conditions, the first ultrasonic sensor is started after Y / S time, where S is the transmission speed of the transmission belt;

[0027] The first ultrasonic sensor dynamically moves along the transmission direction of the transmission belt at a transmission speed S of the transmission belt;

[0028] Obtain a distance L between the first ultrasonic sensor and the transmission belt in the vertical direction, and continuously observe the transmission belt at a fixed angle α; wherein arctan (α) = L / X.

[0029] According to the image-based belt detection method provided by the present disclosure, when the first detection data meets the preset condition, the second ultrasonic sensor is immediately moved along the second direction by a distance X to reach the second position;

[0030] After reaching the second position, the second ultrasonic sensor is activated;

[0031] The distance between the second ultrasonic sensor and the center of mass is obtained, the dynamic observation angle is adjusted to the center of mass and the transmission belt is continuously observed, so that the observation center of the second ultrasonic sensor is always located at the center of mass.

[0032] In a second aspect, the present disclosure further provides an image-based belt detection device, comprising:

[0033] A weight sensor module, used for acquiring first detection data of the weight sensor and calculating the centroid of the first detection data;

[0034] A fusion module is used to fuse the observation data of the first ultrasonic wave and the second ultrasonic wave when the first detection data meets the preset conditions, wherein the fusion includes:

[0035] Convert K first ultrasonic observation data into depth images and sort them according to observation timestamps, where K is a natural number greater than 1; determine motion vectors of key feature points between each two adjacent depth images, calculate difference information of K-1 motion vectors, extract a first boundary contour based on the difference information, and obtain the directivity of the first boundary contour, where the directivity refers to the longitudinal extension direction of the first boundary contour;

[0036] Taking the centroid of the first detection data as the starting point of the second ultrasonic observation data, determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour, identifying the boundary pixel points, and obtaining the second boundary contour;

[0037] The determination module is used to calculate the ratio of the intersection area of ​​the first boundary contour and the second boundary contour to the union area, and when the ratio is greater than a threshold, the intersection area is used as the target area.

[0038] Compared with the prior art, the present disclosure relates to an image-based belt detection method, which includes: obtaining first detection data of a weight sensor, calculating the centroid of the first detection data; when the first detection data meets a preset condition, fusing the observation data of the first ultrasonic wave and the second ultrasonic wave, wherein the fusion includes: converting K frames of first ultrasonic wave observation data into a depth image; determining the motion vector and difference information of key feature points between each two adjacent depth images, and extracting the first boundary contour based on the difference information; determining the gradient change of the second ultrasonic wave observation data along the directivity of the first boundary contour, and extracting the second boundary contour; calculating the ratio of the intersection area of ​​the first boundary contour and the second boundary contour to the union area, and when the ratio is greater than a threshold, taking the intersection area as the target area. By adopting the above scheme, the belt condition can be detected and analyzed with high precision and high reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present disclosure, the drawings required for use in the embodiments or prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 is a flow chart of an image-based belt detection method provided by the present disclosure;

[0041] Figure 2 is a side view of an image-based belt detection sensor provided by the present disclosure;

[0042] Figure 3 is a top view of an image-based belt detection sensor provided by the present disclosure;

[0043] Figure 4 is a schematic diagram of an image-based belt detection device provided by the present disclosure;

[0044] Figure 5 A schematic diagram of the electronic device provided in the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0046] In industrial production, transmission belts are widely used to transport materials such as coal and lime. The materials are easily entrained with various sharp foreign objects, such as iron wires, glass fragments, metal blocks, etc. If these foreign objects are not detected and removed in time, they will enter the belt transmission system along with the materials.

[0047] In actual material transportation scenarios, at the initial transportation position of the transmission belt, due to the accumulation of materials and the influence of initial friction, foreign matter is very easy to embed into the belt. And as the belt continues to move, the embedded foreign matter will continue to move inside the belt, and the tear will gradually accumulate and expand, forming a tear that gradually expands from the initial position. Therefore, the initial position of the transmission belt is the position where the belt is most likely to tear. Detecting the tear at the initial position of the transmission belt can detect the tear in time at the starting position of the tear, thereby avoiding the tear from expanding with the belt transportation, and timely warning to ensure the safe operation of the transmission system.

[0048] Figure 1 is a flow chart of an image-based belt detection method provided by the present disclosure, such as Figure 1 As shown, the method includes:

[0049] Step 1: Acquire first detection data of the weight sensor and calculate the centroid of the first detection data;

[0050] Specifically, in a material transmission system, when a tear occurs at the initial position of the transmission belt, the material will leak from the tear and fall onto the weight sensor, so that the weight sensor generates the first detection data; Figure 2 As shown, the weight sensor is located below the initial position of the transmission belt.

[0051] Step 2: When the first detection data meets the preset conditions, the observation data of the first ultrasonic wave and the second ultrasonic wave are fused;

[0052] Unlike detection methods based on pure vision, ultrasonic sensors are not affected by visual factors such as lighting conditions, color changes, and shadows. They can work stably in various complex environments, such as at night, in foggy days, and in dusty environments, which greatly enhances the environmental adaptability of the method.

[0053] Specifically, once it is detected that the first detection data meets the preset conditions, such as a sudden increase in weight, it will be immediately determined that the longitudinal tear of the belt may occur. At this time, it is necessary to quickly locate the specific location where the abnormality occurs, such as Figure 3 The working environment of the factory material transportation process is very complex. Sometimes debris will fall onto the weight sensor, which is easy to generate noise. Therefore, in order to accurately judge the longitudinal tear of the belt, the preliminary abnormal position information provided by the weight sensor can quickly determine the area where the problem may exist, thereby guiding the movement direction of the ultrasonic sensor and improving the accuracy and efficiency of abnormality detection. Specifically, after obtaining the potential preliminary abnormal position information of the weight sensor, the observation data of the first ultrasonic sensor and the second ultrasonic sensor are further integrated to accurately detect the detailed tear position information of the transmission belt.

[0054] The fusion includes:

[0055] (1) Extracting the first boundary contour of the first ultrasonic wave: converting K frames of first ultrasonic wave observation data into depth images and sorting them according to the observation timestamp, where K is a natural number greater than 1; determining the motion vectors of the key feature points between each two adjacent depth images, calculating the difference information of K-1 motion vectors, extracting the first boundary contour based on the difference information, and obtaining the directivity of the first boundary contour; the directivity refers to the longitudinal extension direction of the belt tear area in the image.

[0056] In one embodiment, taking the centroid of the first detection data as a starting point, image feature points whose weight information detected by the weight sensor reaches a first threshold within a preset distance are selected as key feature points;

[0057] For each two adjacent depth images, match the key feature points in the current image with the corresponding key feature points in the previous image;

[0058] Calculate the angular change of the motion vector of the matched key feature point between each two adjacent depth images;

[0059] The difference information is calculated by the angular change of the motion vector.

[0060] In this step, through continuous observation and conversion into depth images, subtle changes in the belt surface (especially potential tearing areas) can be captured. The first boundary contour is extracted and its directionality is determined by using motion vector analysis of key feature points between adjacent depth images. This dynamic boundary extraction method significantly improves the detection accuracy of abnormal conditions such as belt tearing.

[0061] (2) Extracting the second boundary contour of the second ultrasonic wave: taking the centroid of the first detection data as the starting point of the second ultrasonic wave observation data, determining the gradient change of the second ultrasonic wave observation data along the directivity of the first boundary contour, identifying the boundary pixel points, and extracting the second boundary contour.

[0062] In one implementation, the ultrasonic intensity variance value between each observation data and the observation data in its 8-neighborhood in the second ultrasonic observation data is calculated to obtain the gradient change.

[0063] Specifically, the observation data provided by the second ultrasonic sensor is scan lines arranged in time series, each scan line containing reflection intensity information of the ultrasonic signal; in the ultrasonic observation data, the gradient represents the rate of change of the ultrasonic signal intensity in space, and the gradient is a vector pointing to the direction in which the signal intensity increases fastest;

[0064] The horizontal gradient is calculated by comparing the ultrasonic signal strength of adjacent scan lines. The vertical gradient is calculated by comparing the ultrasonic signal strength of adjacent scan lines. For each scan line, the horizontal gradient and the vertical gradient are combined into a vector to represent the gradient at that point. The vector contains the magnitude and direction information of the gradient.

[0065] When the gradient value of a scan line exceeds the set value, it is considered that the point may be on the boundary.

[0066] The fusion process combines the observation data of the first and second ultrasonic sensors to achieve multi-source data verification analysis of the belt status. This multiple verification mechanism enhances the ability to identify abnormal situations and reduces the risk of false alarms and missed alarms.

[0067] Step 3: Calculate the ratio of the intersection area of ​​the first boundary contour and the second boundary contour to the union area. When the ratio is greater than a threshold, take the intersection area as the target area.

[0068] Specifically, by calculating the intersection-over-union ratio and setting a threshold, the noise area with a low degree of overlap with the real target area can be effectively filtered out, thereby improving the accuracy of target detection. By screening the ratio, the computational complexity can also be reduced and the overall performance of the algorithm can be improved.

[0069] Compared with the prior art, the present invention obtains the first detection data of the weight sensor and calculates the centroid of the first detection data; when the first detection data meets the preset conditions, the first ultrasonic wave and the second ultrasonic wave observation data are fused, wherein the fusion includes: converting K frames of the first ultrasonic wave observation data into a depth image; determining the motion vector and difference information of the key feature points between each two adjacent depth images, and extracting the first boundary contour based on the difference information; determining the gradient change of the second ultrasonic wave observation data along the directivity of the first boundary contour, and extracting the second boundary contour; calculating the ratio of the intersection area of ​​the first boundary contour and the second boundary contour to the union area, and when the ratio is greater than a threshold, the intersection area is used as the target area. By adopting the above scheme, the belt condition can be detected and analyzed with high precision and high reliability.

[0070] In one embodiment, the step of obtaining the first detection data of the weight sensor in step 1, wherein the weight sensor is located below the initial position of the transmission belt in the form of an array unit, includes:

[0071] like Figure 3 As shown, in order to accurately monitor the weight change when the belt is torn and the material leaks to the weight sensor, weight sensors arranged in the form of array units are used. The weight sensors are installed on a bracket or base under the transmission belt to ensure that they can accurately sense the specific location of the material leakage. The arrangement of the array unit is determined by the width of the belt, the distribution characteristics of the material, and the detection requirements of the system. By combining multiple sensors at a certain spacing and arrangement, a complete detection area can be formed to cover the initial position of the entire transmission belt.

[0072] The weight sensor is specially used to measure the weight of an object. It is not affected by various factors such as lighting conditions, object color and texture, and has high-precision measurement capabilities, and can accurately measure the weight of an object. The present disclosure sets a detection device at the initial position, i.e., the position where tearing is most likely to occur, so that the tearing can be detected in time when a very small tear just occurs, thereby avoiding the tear from expanding with the belt transportation, giving a timely warning, and ensuring the safe operation of the transmission system.

[0073] like Figure 3 As shown, the weight sensor is located below the initial position of the transmission belt in the form of an array unit; the array unit is a weight sensor arranged in a two-dimensional array of M×N, the width of the array unit is equal to the width of the transmission belt, and M and N are natural numbers;

[0074] The first detection data includes weight information and corresponding position information of each weight sensor.

[0075] In one implementation, when the first detection data meets the preset condition, the first detection data meeting the preset condition includes:

[0076] Traversing the sensor array: dividing the sensor array unit into multiple blocks, setting priorities for the multiple blocks respectively, wherein the priority of the central area of ​​the transmission belt is higher than that of other areas; using a multi-threaded method and traversing multiple blocks of the sensor array unit according to the priority level;

[0077] When the weight information detected by the traversed T adjacent weight sensors all reaches the first threshold, the centroids of the T adjacent weight sensors are obtained, and the X and Y coordinates of the centroids in the array unit are calculated.

[0078] By traversing the sensor array in a multi-threaded manner, the detection data of multiple areas can be processed in parallel, significantly improving the speed and efficiency of data processing. This parallel processing mechanism can complete the detection of the entire belt in a short time, thereby improving real-time performance.

[0079] Specifically, in order to accurately locate the abnormal area, the centroid of T adjacent sensors is first calculated. The centroid is the geometric center of these sensors in the array unit, which can provide a center point representing the entire abnormal area. The process of calculating the centroid includes weighting the position coordinates of each sensor with the weight information it detects. The size of the weight is usually proportional to the detected weight, that is, the greater the detected weight, the greater the weight of the sensor position coordinate. Then sum and divide by the total weight to get the coordinates of the center of mass.

[0080] In one embodiment, the movement of the ultrasonic sensor includes:

[0081] like Figure 3 As shown, the triangle in the figure represents the preliminary abnormal position information provided by the weight sensor. At this time, the first ultrasonic sensor and the second ultrasonic sensor are started to perform synchronous detection based on the preliminary abnormal position information.

[0082] Firstly, the first ultrasonic sensor is dynamically moved along a first direction, where the first direction is the belt transmission direction. During the movement, the transmission belt is continuously observed at a fixed angle, which can ensure that the observation center point is always focused on the initial abnormal position, thereby improving the accuracy and efficiency of the observation.

[0083] At the same time, the second ultrasonic sensor is moved to a second position along a second direction perpendicular to the first direction and then started. The dynamic observation angle is adjusted according to the transmission speed of the transmission belt and the transmission belt is continuously observed. This can also ensure that the observation center point of the first ultrasonic sensor is always focused on the initial abnormal position, effectively improving the accuracy of the observation.

[0084] Specifically, the moving process of the first ultrasonic sensor includes: when the first detection data meets the preset conditions, after Y / S time, the first ultrasonic sensor is started, where S is the transmission speed of the transmission belt;

[0085] The first ultrasonic sensor dynamically moves along the transmission direction of the transmission belt at a transmission speed S of the transmission belt;

[0086] Obtain a distance L between the first ultrasonic sensor and the transmission belt in the vertical direction, and continuously observe the transmission belt at a fixed angle α; wherein arctan (α) = L / X.

[0087] Specifically, when the system detects that the first detection data meets the preset conditions, it waits for a period of time Y / S (S is the transmission speed of the transmission belt) before starting the first ultrasonic sensor. The waiting period is to ensure that the belt tear position moves to the observation center of the first ultrasonic sensor, and then the belt and the first ultrasonic sensor move at the same speed to keep the observation center of the ultrasonic sensor always at the tear center, thereby ensuring that the observation area of ​​the ultrasonic sensor always covers the area where the abnormality may exist.

[0088] Specifically, the moving process of the second ultrasonic sensor includes: when the first detection data meets the preset condition, immediately moving the second ultrasonic sensor along the second direction by a distance X to reach the second position;

[0089] After reaching the second position, the second ultrasonic sensor is activated;

[0090] The distance between the second ultrasonic sensor and the center of mass is obtained, the dynamic observation angle is adjusted to the center of mass and the transmission belt is continuously observed, so that the observation center of the second ultrasonic sensor is always located at the center of mass.

[0091] An image-based belt detection device provided by the present disclosure is described below. The detection system described below and the detection method described above can refer to each other.

[0092] like Figure 4 As shown, an image-based belt detection device comprises:

[0093] A weight sensor module, used for acquiring first detection data of the weight sensor and calculating the centroid of the first detection data;

[0094] A fusion module is used to fuse the observation data of the first ultrasonic wave and the second ultrasonic wave when the first detection data meets the preset conditions, wherein the fusion includes:

[0095] Convert K first ultrasonic observation data into depth images and sort them according to observation timestamps, where K is a natural number greater than 1; determine the motion vectors of key feature points between each two adjacent depth images, calculate the difference information of K-1 motion vectors, extract the first boundary contour based on the difference information, and obtain the directivity of the first boundary contour;

[0096] Taking the centroid of the first detection data as the starting point of the second ultrasonic observation data, determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour, identifying the boundary pixel points, and obtaining the second boundary contour;

[0097] The determination module is used to calculate the ratio of the intersection area and the union area of ​​the first boundary contour and the second boundary contour, and when the ratio is greater than a threshold, the intersection area is used as the target area.

[0098] An electronic device provided in an embodiment of the present application is Figure 5 As shown, the electronic device includes a processor 502 and a memory 501, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0099] See also Figure 5 The electronic device further includes: a bus 503 and a communication interface 504, a processor 502, a communication interface 504 and a memory 501 are connected via the bus 503; the processor 502 is used to execute an executable module stored in the memory 501, such as a computer program.

[0100] The memory 501 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 504 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0101] The bus 503 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0102] Among them, the memory 501 is used to store programs, and the processor 502 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 502 or implemented by the processor 502.

[0103] The processor 502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 502. The above processor 502 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 501, and the processor 502 reads the information in the memory 501 and completes the steps of the above method in combination with its hardware.

[0104] Corresponding to the above-mentioned image-based belt detection method, an embodiment of the present application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer-readable storage medium stores computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned steel coil center measurement method.

[0105] The image-based belt detection device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0106] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0107] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0110] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the image-based belt detection method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0111] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0112] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A belt detection method based on an image, characterized in that: The following steps are involved: Acquire first detection data of the weight sensor, and calculate the centroid of the first detection data; Wherein, the obtaining of the first detection data of the weight sensor comprises: The weight sensor is located below the initial position of the transmission belt in the form of an array unit; The array unit is a weight sensor arranged in a two-dimensional array of M×N. The width of the array unit is equal to the width of the transmission belt. M and N are natural numbers. The first detection data includes weight information and corresponding position information of each weight sensor; When the first detection data meets the preset condition, fusing the observation data of the first ultrasonic wave and the second ultrasonic wave; The first detection data meeting the preset condition includes: Traversing the sensor array: dividing the sensor array unit into multiple blocks, setting priorities for the multiple blocks respectively, wherein the priority of the central area of ​​the transmission belt is higher than that of other areas; using a multi-threaded method and traversing multiple blocks of the sensor array unit according to the priority level; When the weight information detected by the traversed T adjacent weight sensors all reaches the first threshold, the centroids of the T adjacent weight sensors are obtained, and the X and Y coordinates of the centroids in the array unit are calculated; Wherein, the fusion includes: Convert K first ultrasonic observation data into depth images and sort them according to observation timestamps, where K is a natural number greater than 1; determine motion vectors of key feature points between each two adjacent depth images, calculate difference information of K-1 motion vectors, extract a first boundary contour based on the difference information, and obtain the directivity of the first boundary contour, where the directivity refers to the longitudinal extension direction of the first boundary contour; Taking the centroid of the first detection data as the starting point of the second ultrasonic observation data, determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour, identifying the boundary pixel points, and extracting the second boundary contour; Calculating a ratio of an intersection area of ​​the first boundary contour and a second boundary contour to a union area, and when the ratio is greater than a threshold, taking the intersection area as a target area; When the first detection data meets the preset conditions, the first ultrasonic sensor is started after Y / S time, where S is the transmission speed of the transmission belt; The first ultrasonic sensor dynamically moves along the transmission direction of the transmission belt at a transmission speed S of the transmission belt; Obtain a distance L between the first ultrasonic sensor and the transmission belt in the vertical direction, and continuously observe the transmission belt at a fixed angle α; wherein arctan (α) = L / X; When the first detection data meets the preset condition, the second ultrasonic sensor is immediately moved along the second direction by a distance X to reach the second position; After reaching the second position, the second ultrasonic sensor is activated; The distance between the second ultrasonic sensor and the center of mass is obtained, the dynamic observation angle is adjusted to the center of mass and the transmission belt is continuously observed, so that the observation center of the second ultrasonic sensor is always located at the center of mass.

2. The image-based belt detection method according to claim 1, characterized in that: Determining the motion vector of the key feature point between each two adjacent depth images and calculating the difference information of K-1 motion vectors includes: Taking the centroid of the first detection data as a starting point, selecting image feature points whose weight information detected by the weight sensor within a preset distance reaches a first threshold as key feature points; For each two adjacent depth images, match the key feature points in the current image with the corresponding key feature points in the previous image; Calculate the angular change of the motion vector of the matched key feature point between each two adjacent depth images; The difference information is calculated by the angular change of the motion vector.

3. The image-based belt detection method according to claim 1, characterized in that: Determining the gradient change of the second ultrasonic observation data along the directivity of the first boundary contour includes: The ultrasonic intensity variance value between each observation data and the observation data in the 8-neighborhood in the second ultrasonic observation data is calculated to obtain the gradient change.

4. An electronic device comprising: processor; A memory storing a program, wherein the program comprises instructions which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 3.

5. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer-readable storage medium stores instructions or computer programs, and when the instructions or computer programs are executed on a device, the device executes the method according to any one of claims 1 to 3.

Images