Vision-based trolley wheel positioning and speed detection method

Abstract

The invention relates to a vision-based trolley wheel positioning and speed detection method, which comprises the following steps: acquiring a wheel rotation image as a first image in the running process of a trolley through a camera which is arranged on the side surface of a trolley carriage and is vertically opposite to wheels of the trolley; detecting the first image through a visual target detection algorithm to obtain the position of the wheel; obtaining a wheel ROI region in the first image as a second image; detecting the second image through a visual target detection algorithm to obtain the position of the vehicle nail; repeatedly obtaining and detecting the positions of the same wheel and all vehicle nails on the same wheel after a preset time interval; according to the positions, obtained twice, of the same wheel, the advancing linear speed of the wheel is determined; and determining the angular velocity of the rotation of the wheel according to the positions of all the vehicle nails on the same wheel obtained twice. According to the invention, unmanned monitoring is carried out by using machine vision, so that the safety and accuracy of monitoring are ensured while manpower is greatly saved.

Description

Technical Field

[0001] The present application relates to the technical field of sintering and smelting, and in particular to a vision-based trolley wheel positioning and speed detection method, device and system, computing equipment and storage medium. Background Art

[0002] With the continuous development of industrial intelligence, the role of automation and unmanned operation is becoming more and more popular, especially in extreme working environments such as high temperature and high risk. The use of automated equipment to replace manual operation can not only significantly improve production efficiency and safety, but also effectively reduce the operating costs of enterprises. For example, in the sintering process in the steel manufacturing process, sintering is an important link in steel production. It involves mixing raw materials such as iron ore powder, fuel (such as coal powder), and flux, and sintering them into block materials at high temperatures for subsequent smelting and processing. This process requires not only strict control of parameters such as temperature and pressure, but also real-time monitoring of equipment status to ensure production safety and product quality. However, due to the extremely high temperature inside the sintering furnace (up to 1200°C or more), the traditional manual monitoring method has great safety risks, and it is also difficult to ensure the accuracy and real-time nature of data collection. Summary of the invention

[0003] In view of the above problems in the prior art, the present application provides a vision-based trolley wheel positioning and speed detection method, device and system, computing equipment and storage medium, which uses machine vision for unmanned monitoring, greatly saving manpower while ensuring the safety and accuracy of monitoring.

[0004] In order to overcome the above-mentioned defects, the first aspect of the present application provides a method for wheel positioning and speed detection of a trolley based on vision, comprising:

[0005] The camera is used to obtain a wheel rotation image of the trolley during its travel as a first image, wherein the camera lens is arranged on the side of the trolley compartment vertically facing the wheel of the trolley;

[0006] Detecting the first image using a visual target detection algorithm to obtain the position of the wheel;

[0007] The wheel ROI region in the first image is obtained as the second image using the minimum circumscribed rectangle of the wheel;

[0008] Detecting the second image using a visual target detection algorithm to obtain positions of all studs on the wheel;

[0009] Repeat the above image acquisition and detection steps after a preset time interval to detect the same wheel and the positions of all the studs thereon again;

[0010] Determine the position change of the wheel according to the position of the same wheel in the first images obtained twice, and determine the linear speed of the wheel based on the position change of the wheel and the preset time interval;

[0011] According to the positions of all the studs on the same wheel in the second images obtained twice, the position changes of all the studs are determined, and the angular velocity of the wheel rotation is determined based on the position changes of all the studs and the preset time interval.

[0012] From the above, this application collects images of the trolley wheels, performs detection based on visual target detection images, and then calculates the trolley's straight-line speed and wheel rotation angular velocity, thereby using machine vision to achieve unmanned monitoring, which greatly saves manpower while ensuring the safety and accuracy of monitoring.

[0013] The second aspect of the present application provides a vision-based trolley wheel positioning and speed detection device, comprising:

[0014] An image acquisition module, used to acquire a wheel rotation image of the trolley during its travel as a first image through a camera, wherein the camera lens is arranged on a side of the trolley compartment vertically facing the wheel of the trolley;

[0015] An object detection module, used to detect the first image by using a visual object detection algorithm to obtain the position of the wheel;

[0016] The image acquisition module is also used to: acquire the wheel ROI area in the first image as the second image using the minimum circumscribed rectangle of the wheel;

[0017] The target detection module is also used to: detect the second image by a visual target detection algorithm to obtain the positions of all the studs on the wheel;

[0018] The image acquisition module and the target detection module are also used to: repeat the above image acquisition and detection steps after a preset time interval, and detect the positions of the same wheel and all the studs thereon again;

[0019] A linear speed determination module, used to determine the position change of the wheel according to the position of the same wheel in the first images obtained twice, and determine the linear speed of the wheel based on the position change of the wheel and the preset time interval;

[0020] The angular velocity determination module is used to determine the position change of all the studs on the same wheel according to the positions of all the studs on the same wheel in the second images obtained twice, and determine the angular velocity of the wheel rotation based on the position change of all the studs and the preset time interval.

[0021] The third aspect of the present application provides a vision-based trolley wheel positioning and speed detection system, comprising: a camera, a trolley and the above-mentioned vision-based trolley wheel positioning and speed detection device;

[0022] The camera lenses are arranged vertically on both sides of the carriage facing the wheels, and the images collected by the cameras are provided to the vision-based trolley wheel positioning and speed detection device.

[0023] A fourth aspect of the present application provides a computing device, including:

[0024] processor, and

[0025] A memory having program instructions stored thereon, wherein when the program instructions are executed by the processor, the processor executes any of the methods described in the first aspect above.

[0026] A fifth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, wherein when the program instructions are executed by a computer, the computer implements any of the methods described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a method for wheel positioning and speed detection of a trolley based on vision provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the positions of a camera and a trolley provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a nail captured by a camera provided in an embodiment of the present application;

[0030] Figure 4 It is a structural schematic diagram of a vision-based trolley wheel positioning and speed detection device provided in an embodiment of the present application;

[0031] Figure 5 It is a structural schematic diagram of a computing device provided in an embodiment of the present application.

[0032] It should be understood that the size and shape of each block diagram in the above structural diagram are for reference only and should not constitute an exclusive interpretation of the embodiment of the present invention. The relative position and inclusion relationship between the blocks presented in the structural diagram are only schematic representations of the structural association between the blocks, and do not limit the physical connection method of the embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution provided by the present application is further described below with reference to the accompanying drawings and examples. It should be understood that the system structure and business scenarios provided in the examples of the present application are mainly to illustrate the possible implementation methods of the technical solution of the present application and should not be interpreted as the only limitation on the technical solution of the present application. It is known to those of ordinary skill in the art that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present application. In case of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0035] [Implementation example of a method for detecting wheel positioning and speed of a trolley based on vision]

[0036] This application provides a vision-based trolley wheel positioning and speed detection, such as Figure 1 As shown, including:

[0037] S110: Acquire a wheel rotation image of the trolley during driving as a first image through a camera, wherein a camera lens is arranged on a side of the trolley compartment vertically facing the wheel of the trolley;

[0038] S120: Detecting the first image using a visual target detection algorithm to obtain a position of a wheel;

[0039] S130: Acquire the wheel ROI region in the first image using the minimum circumscribed rectangle of the wheel as the second image;

[0040] S140: Detecting the second image using a visual target detection algorithm to obtain positions of all studs on the wheel;

[0041] S150: repeating the above image acquisition and detection steps after a preset time interval, and detecting again to obtain the positions of the same wheel and all the studs thereon;

[0042] S160: determining a position change of the wheel according to the positions of the same wheel in the first images obtained twice, and determining a linear speed of the wheel based on the position change of the wheel and the preset time interval;

[0043] S170: Determine the position change of all the studs on the same wheel according to the positions of all the studs in the second images obtained twice, and determine the angular velocity of the wheel rotation based on the position change of all the studs and the preset time interval.

[0044] In specific implementation, the trolley refers to a sintering transport trolley, which includes a middle transport carriage and wheels on both sides of the carriage, and the camera lenses are arranged vertically on both sides of the carriage facing the wheels. Figure 2 shown.

[0045] In some embodiments, detecting an image using a visual target detection algorithm to obtain a target location includes:

[0046] The image is detected by a visual target detection algorithm to obtain the position information of the minimum circumscribed rectangle of the target;

[0047] The center position information of the target is obtained based on the position information of the minimum circumscribed rectangle of the target;

[0048] The image is the first image or the second image, and the target is a wheel or a stud.

[0049] In some embodiments, the minimum bounding rectangle is determined as follows:

[0050] P=(X lt , Y lt , Width, Height);

[0051] Among them, X lt , Y lt is the coordinate of the upper left corner of the minimum bounding rectangle of the wheel or stud, Width and Height are the width and height of the bounding rectangle; P is the position information of the minimum bounding rectangle;

[0052] The position of the wheel or the stud is obtained based on the position information of the minimum circumscribed rectangle in the following manner:

[0053]

[0054] Wherein, C is the center position information of the wheel or stud.

[0055] In specific implementation, these embodiments are directed to S120 and S140. Among them, the first image or the second image is detected using the YOLOv5 (You Only LookOnce version 5) target detection algorithm, and the target detection output of YOLOv5 generally includes bounding box information of each detected object, which is given in the form of center point coordinates (x, y), width w and height h, all of which are normalized values ​​relative to the input image size. This information actually defines a minimum bounding rectangle to enclose the detected object.

[0056] In order to get the position information P of the wheel in the image from YOLOv5s, you need to know the bounding box coordinates output by the model. Assume that you already have the detection results of the wheel by YOLOv5s, and these results are given in standardized coordinates, that is, values ​​between 0 and 1, indicating the proportional position relative to the input image size.

[0057] Here are the steps for calculating the minimum bounding rectangle P of the wheel from the output of YOLOv5s for S120:

[0058] 1. Get YOLOv5s output: First, make sure you have run the YOLOv5s model and get the output containing the wheel detection results. The output usually contains the center point coordinates (x, y) of the bounding box and the width w and height h of the bounding box, all of which are normalized values ​​relative to the input image size.

[0059] 2. Convert to pixel coordinates: Convert these normalized coordinates to actual pixel coordinates. This usually involves multiplying by the width and height of the image. For example, if the width of the image is W and the height is H, then the actual pixel coordinates can be calculated like this:

[0060] The pixel coordinate x of the center point center =x*W

[0061] The pixel coordinate y of the center point center =y*H

[0062] 3. Calculate the coordinates of the upper left corner: With the pixel coordinates of the center point, the coordinates of the upper left corner of the bounding box can be calculated. Since YOLOv5 outputs the center and width / height of the bounding box, the coordinates of the upper left corner can be obtained by subtracting half of the width and height:

[0063] X coordinate of the upper left corner lt =x center -[(w*W) / 2]

[0064] Y coordinate of the upper left corner lt =y center -[(h*H) / 2]

[0065] 4. Calculate the width and height of the bounding box: The width and height of the bounding box can be obtained directly from the model output, and then multiplied by the width and height of the image to convert to pixel units:

[0066] Width = w*W

[0067] Height = h*H

[0068] Finally, the position information P of the minimum circumscribed rectangle of the wheel can be expressed as: P = (X lt , Ylt , Width, Height).

[0069] Then the position of the wheel is calculated based on the position information P of the minimum circumscribed rectangle of the wheel:

[0070]

[0071] Wherein, C is the center position information of the wheel.

[0072] In the specific implementation, for S130, the wheel position information is obtained, and the wheel ROI area is established on the first image based on the minimum circumscribed rectangle of the wheel. This area is used as the area to be detected, and then the yolov5 algorithm is used in S140 to detect the stud target. The studs are placed in a circle around the wheel, such as Figure 3 The position information of the minimum circumscribed rectangle of the nails is saved in the P format mentioned above, and the center position of each nail is saved in the C format mentioned above.

[0073] The specific processes of S130 and S140 are described in detail below:

[0074] Step 1: Determine the ROI area of ​​the wheel

[0075] 1. Extract ROI area: Extract the ROI area containing the wheel according to P. This can be achieved by cropping the original image. The cropping range is (X lt , Y lt ) to (X lt +Width,Y lt +Height). Where X lt and Y lt is the coordinate of the upper left corner of the minimum circumscribed rectangle of the wheel, and Width and Height are the width and height of the minimum circumscribed rectangle of the wheel.

[0076] Step 2: Car nail target detection

[0077] 1. Use the YOLOv5 algorithm: Use the extracted wheel ROI area as the new input image (second image), and use the YOLOv5 algorithm again for target detection. This time the target is the studs on the wheel.

[0078] 2. Save the detection results: The YOLOv5 algorithm will return the location information of the nails in the ROI area, which is also given in the form of the minimum bounding rectangle of the nails. The saving format is still P lt_roi =(X lt_roi , Y lt_roi , Width lt_roi ,Height lt_roi ), where X lt_roi and Ylt_roi is the coordinate relative to the upper left corner of the ROI area, Width lt_roi ,Height lt_roi is the width and height of the minimum circumscribed rectangle of the nail.

[0079] Step 3: Calculate the center position of the nail

[0080] For each nail, the position information P of the minimum bounding rectangle of the nail can be used to calculate the lt_roi Calculate the position of the nail: C lt_roi =(X lt_roi +Width lt_roi / 2,Y lt_roi +Height lt_roi / 2).

[0081] Example Description

[0082] Assume that in a certain detection, the wheel position information is P = (100, 150, 200, 150), which means that the coordinates of the upper left corner of the wheel are (100, 150), the width is 200 pixels, and the height is 150 pixels. Three studs are detected from this area, and their position information is:

[0083] First nail: P1 = (50, 50, 20, 20)

[0084] The second nail: P2 = (120, 70, 15, 15)

[0085] The third nail: P3 = (170, 100, 25, 25)

[0086] Calculate the center coordinates of the three studs:

[0087] The center point of the first nail: C1 = (50 + 20 / 2, 50 + 20 / 2) = (60, 60)

[0088] The second nail center point: C2 = (120 + 15 / 2, 70 + 15 / 2) = (137.5, 87.5)

[0089] The third nail center point: C3 = (170 + 25 / 2, 100 + 25 / 2) = (192.5, 112.5)

[0090] In this way, the position information of the wheel and its studs can be accurately detected and recorded, providing data support for subsequent motion analysis.

[0091] In specific implementation, for S150, a fixed number of frames is set, and the position information of the same wheel in the two images before and after is collected and recorded as P1 and P2. Based on the detected information of P1 and P2, the wheel center positions C1 and C2 are calculated respectively. Then, two second images are obtained by cropping the circumscribed minimum rectangle of the same wheel in the two first images before and after the collection, and the obtained information of the position of the nails in the two second images is recorded as P3 and P4. Based on the detected information of P3 and P4, the center positions of the nails C3 and C4 are calculated respectively.

[0092] In specific implementation, for S160, based on the obtained position information of C1 and C2, the lateral absolute position change in the pixel is calculated to obtain the position offset difference ΔP, and then combined with the camera focal length and other parameters to map to the real world coordinate system to calculate the movement amount ΔX of the wheel. Then, based on the sampling interval and ΔX, the linear speed V of the wheel is obtained. l Based on this, in some embodiments, the position change of the wheel is determined according to the position of the same wheel in the first image obtained twice according to the following formula:

[0093] ΔX=(d_pix×ΔP)×z / focla

[0094] Wherein, z is the distance between the camera and the center of the wheel facing it, focal is the focal length of the camera lens, ΔX is the actual distance traveled by the wheel, d_pix is ​​the pixel size of the camera, ΔP is the lateral absolute position change of the center position of the wheel in the pixel coordinate system in the first image obtained twice, and ΔX is the position change of the wheel (i.e., the movement amount of the wheel);

[0095] The linear speed of the wheel is determined based on the position change of the wheel and the preset time interval according to the following formula:

[0096] V l =ΔX / t

[0097] Among them, V l is the linear speed of the wheel, and t is the preset time interval.

[0098] Specifically, according to the wheel center positions C1 and C2 calculated in S120, their lateral absolute position changes in the pixel coordinate system, that is, the position offset difference ΔP, are calculated. This difference represents the position change of the wheel in the lateral direction (usually the X-axis direction) in the two images.

[0099] ΔP=|C1x-C2x|

[0100] Where C1x is the X coordinate in C1 and C2x is the X coordinate in C2.

[0101] Example Description

[0102] Assumption: ΔP = 50 pixels (ie, the lateral displacement of the center point of the wheel in the two images).

[0103] dpix = 0.005 mm (assuming the pixel size of the camera sensor).

[0104] focal=10mm (focal length of the camera lens).

[0105] z = 1000 mm (distance between camera and wheel center).

[0106] t = 0.5 seconds (time interval between two image acquisitions).

[0107] Calculate ΔX:

[0108] ΔX = (0.00510) × 50 × 1000 = 0.0005 × 50 × 1000 = 25 ΔX = (100.005) × 50 × 1000 = 0.0005 × 50 × 1000 = 25 mm (or 0.025 m)

[0109] Calculate V l :

[0110] V l =0.0250.5=0.05Vl=0.50.025=0.05 m / s

[0111] Through these steps, the movement amount and linear speed of the trolley wheels can be accurately calculated, thus providing necessary data support for applications such as automation control and condition monitoring.

[0112] In specific implementation, for S170, all the positions of the studs obtained in the second image are circle-fitted according to the positions, the position changes in the two steps are compared to obtain the angle difference, and the angular velocity of the wheel rotation is calculated in combination with the time. Based on this, in some embodiments, the position changes of all the studs on the same wheel in the second images obtained twice are determined, and the angular velocity of the wheel rotation is determined based on the position changes of all the studs and the preset time interval, including:

[0113] Use the least square method to perform circle fitting on the positions of all the nails detected each time to obtain the center position information of the nails;

[0114] Calculating the angle change of the same nail in the second images obtained twice according to the center position information of the nail and the center position information of all the nails;

[0115] The angular velocity of the wheel rotation is determined based on the angle change of the same stud in the second images obtained twice and the preset time interval.

[0116] In specific implementation, in some embodiments, the least square method is used to perform circle fitting on the positions of all the nails detected each time according to the following formula to obtain the center position information of the nails:

[0117] f=∑((x i -x c ) 2 +(y i -y c ) 2 -R 2 ) 2

[0118] Where f is the residual sum of squares between all detected nail positions and the fitted circle, x i and i is the center coordinate of the nail; x c and c are the coordinates of the center of the fitted circle; R is the radius.

[0119] Specifically, in order to describe in detail how to use the least squares method to fit the circle of the stud coordinates and obtain the coordinates of the center of the circle, the following steps can be followed:

[0120] 1. Collect the coordinates of the nails

[0121] Assume that the YOLOv5 algorithm has been used to obtain the coordinate sets of the two detected nails. Let the coordinate set of the nails obtained by the first detection be A = {(x 1i ,y 1i )}, the second detection of the nail coordinate set is B = {(x 2i ,y 2i )}.

[0122] 2. Circle fitting using the least squares method

[0123] Use the least squares method to fit the coordinates of the nails to find the center coordinates of the best fitting circle (x c ,y c ) and radius R. The least squares method is a commonly used method to find a set of parameters that minimizes the sum of squared residuals.

[0124] The objective function of the least squares method is: f = ∑((x i -x c ) 2 +(y i -y c ) 2 -R 2 ) 2

[0125] Among them, x i and iare the coordinates of the center position of the nail.

[0126] x c and c are the coordinates of the center of the fitted circle.

[0127] R is the radius of the fitted circle.

[0128] f is an error function or objective function, which represents the sum of squared deviations between all detected stud positions and the fitted circle. This function is used to measure the fitted circle (centered by the circle (x c ,y c ) and the radius R define the degree of difference between the actual observed nail position. Specifically, the smaller the value of f, the closer the fitted circle is to the actual observed nail position, that is, the smaller the error. The goal of the least squares method is to find a set of parameters (x c ,y c , R), so that f reaches the minimum value. In other words, it is to find such a center and radius that the sum of the square differences of the distances from all nail positions to the center of the circle is minimized.

[0129] 3. Calculate the center coordinates and radius

[0130] In order to solve the above objective function, numerical optimization methods (such as gradient descent method, Newton method, etc.) can be used to find the minimum f(x c ,y c , R) parameters (x c ,y c , R). The specific steps are as follows:

[0131] (1) Initialization parameters: Select an initial center coordinate (x c (0) ,y c (0) ) and radius R(0).

[0132] (2) Calculate the gradient: Calculate the objective function with respect to x c ,y c and the partial derivatives of R.

[0133] (3) Update parameters: Use gradient descent or other optimization methods to update parameters:

[0134]

[0135] Here, α is the step size (learning rate).

[0136] (4) Iterate until convergence: Repeat steps (2) and (3) until the parameter change is less than a certain threshold or the maximum number of iterations is reached.

[0137] In some embodiments, before calculating the angle change of the same nail in the two second images obtained, the method further includes:

[0138] Taking the positions of all the nails detected for the first time as a reference, the Euclidean distances with the positions of all the nails obtained for the second time are calculated respectively, and the group with the closest Euclidean distance is regarded as the same nail.

[0139] In some embodiments, taking the positions of all the nails detected for the first time as a reference, respectively calculating the Euclidean distances with the positions of all the nails obtained for the second time, and considering the group with the closest Euclidean distance as the same nail, including:

[0140] For each element a i ∈A, calculate a i With each element b in B j The Euclidean distance of:

[0141]

[0142] Find the number of cells in B that use d(a i , b j )The smallest element:

[0143]

[0144] Among them, A represents the location information set of all the nails detected for the first time, a i It is represented by the elements of A; B represents the location information set of all the studs detected for the first time, b j represents the elements of B; d represents the Euclidean distance between elements; b closest It is expressed as the minimum Euclidean distance of an element in B that is considered the same as an element in A.

[0145] In specific implementation, when obtaining the center coordinates of the best fitting circle and performing subsequent calculation of the angle change of the same nail in the second image obtained twice based on the center position information of the nail and the center position information of all the nails, it is also necessary to calculate the Euclidean distance between the nails in the two previous and next detection results to find the group with the closest distance, so as to determine the same nail in the two previous and next detections. This ensures that the position change of the same nail can be correctly tracked in subsequent processing, which is very important for calculating parameters such as the angle change and movement distance of the nail. By minimizing the Euclidean distance, the impact caused by detection errors can be reduced and the accuracy of matching can be improved.

[0146] In some embodiments, calculating the angle change of the same nail in the second images obtained twice according to the center position information of the nail and the center position information of all the nails includes:

[0147] The first vector v1 and the second vector v2 from the same nail to the center of the circle are calculated twice:

[0148] v1=(x i 1-x c 1, y i 1-y c 1)

[0149] v2=(x i 2-x c 2, y i 2-y c 2)

[0150] Calculate the dot product of the first vector v1 and the second vector v2:

[0151] v1·v2=(x i 1-x c 1)×(x i 2-x c 2)+(y i 1-y c 1)×(y i 2-y c 2)

[0152] Calculate the magnitude of the first vector v1 and the second vector v2:

[0153]

[0154] Calculate the cosine of the angle as the ratio of the dot product of the first vector v1 and the second vector v2 to the product of the magnitude of the first vector and the magnitude of the second vector:

[0155]

[0156] Calculate the angle θ and convert it to radians:

[0157]

[0158] Among them, x i 1.y i 1 is the center position coordinate of any nail obtained for the first time; x i 2.y i 2 is the center coordinate of the same nail obtained for the second time; x c 1 and y c 1 is the coordinate of the center of the fitting circle obtained for the first time, x c 2 and y c 2 is the coordinate of the center of the fitting circle obtained for the first time, and the angle θ is the angle change of the same nail in the second image obtained twice.

[0159] In specific implementation, after determining the same nail in two consecutive detections, subsequent calculation is performed based on the center position information of the same nail in the two consecutive detections and the center position information of the nail.

[0160] Example Calculation

[0161] Assume that the coordinates of the circle center obtained by the first detection are (x c 1, y c 1)=(200,200), the coordinates of the center of the circle obtained by the second detection are (x c 2, y c 2) = (210, 210).

[0162] For a matching stud, the center coordinates obtained by the first detection are (x i 1, y i 1)=(220,220), the center coordinates obtained by the second detection are (x i 2, y i 2) = (230, 230).

[0163] Calculate vectors v1 and v2: v1 = (220-200, 220-200) = (20, 20) v2 = (230-210, 230-210) = (20, 20)

[0164] Calculate the dot product: v1·v2=20×20+20×20=800

[0165] Calculation module:

[0166] Calculate the cosine of the angle:

[0167] Calculate the angle: θ = arccos(1) = 0, which means that the two vectors v1 and v2 are completely in the same direction, that is, the angle between them is zero degrees. In other words, the two vectors have the same direction and no angle difference. This means that in the two tests, the position of the stud relative to the center of the circle did not rotate, that is, their directions are exactly the same. However, in actual applications, this situation may be rare. Usually, due to detection errors or other factors, there will be a certain angle between the two vectors, even if this angle is very small.

[0168] When the wheel is running normally, the position of the stud relative to the center of the wheel should be stable, that is, there should be no significant rotational changes. If the position of the stud relative to the center of the wheel is found to have rotated, this may mean that there are some problems:

[0169] (1) Abnormal vibration: The rotational change of the stud position may be caused by abnormal vibration due to wheel imbalance.

[0170] (2) Wear or damage: Changes in the position of the nails may be caused by wear or damage, and it is necessary to check whether there is looseness or breakage.

[0171] (3) Installation problem: If the position of the nails is unstable, it may be caused by improper installation and needs to be re-calibrated.

[0172] Therefore, the purpose of calculating the angle change of the same nail in the second image obtained twice is:

[0173] (1) Quantifying angle change: By calculating the angle change of the same nail in two images, the rotation of the nail can be quantified.

[0174] (2) Monitoring state changes: Angle changes can reflect changes in the state of the wheel, which is very useful for monitoring the state of the wheel.

[0175] (3) Providing reference data: The data of angle changes can provide a reference for subsequent analysis, such as determining whether any abnormal situation has occurred.

[0176] Through the above steps, the angle change of the same nail in the two images can be effectively calculated, and the necessary data support can be provided for further analysis.

[0177] In some embodiments, the angular velocity of the wheel rotation is determined based on the angle change of the same stud in the two obtained second images and the preset time interval according to the following formula:

[0178]

[0179] Wherein, t is the preset time interval.

[0180] During the specific implementation, assume the following specific data:

[0181] The angle change θ = 0.5 radians.

[0182] The video frame rate is FPS = 30 frames per second.

[0183] The time interval between two adjacent frames is: t = 1 / 30 seconds

[0184] Calculate the angular velocity: ω = θ / t = 0.5 / 1 / 30 = 0.5 × 30 = 15 rad / s

[0185] The purpose of calculating the angular velocity of the wheel rotation is:

[0186] (1) Quantifying rotation speed: By calculating the angular velocity, the speed at which the wheel rotates can be quantified, which is very useful for monitoring the status of the wheel.

[0187] (2) Real-time monitoring: Angular velocity can be used to monitor the rotation of the wheels in real time, helping to determine whether there is abnormal rotation or a fault, thereby ensuring smooth operation of the vehicle.

[0188] (3) Data analysis: Angular velocity data can be used for further data analysis. By looking at the changing trend of angular velocity, it can assist in diagnosing whether the wheel is rotating at a constant speed, whether there is shaking, whether there is wear, looseness, and other problems.

[0189] Through the above steps, the angular velocity of the wheel can be effectively calculated, and provide data support for further analysis and application.

[0190] In summary, the vision-based trolley wheel positioning and speed detection method proposed in this application can achieve the following beneficial effects:

[0191] (1) High-precision detection:

[0192] Through the visual target detection algorithm, the position of the wheel can be accurately located and the ROI area of ​​the wheel can be obtained, thereby improving the detection accuracy.

[0193] Detecting and recording the positions of all studs on the wheel allows for a more detailed analysis of the wheel's condition.

[0194] (2) Real-time monitoring:

[0195] By setting a fixed frame interval for repeated acquisition and detection, real-time monitoring of the wheel rotation status can be achieved and potential problems can be discovered in a timely manner.

[0196] (3) Multi-dimensional analysis:

[0197] By mapping the position change into the real physical coordinate system, the linear speed of travel can be calculated, providing information on the wheel travel speed.

[0198] The angular velocity is calculated by circle fitting and position change, which provides information on the wheel rotation speed and allows a comprehensive understanding of the wheel's motion state.

[0199] (4) High degree of automation:

[0200] The entire process is highly automated, reducing the need for human intervention and improving efficiency and reliability.

[0201] On industrial production lines, the movement status of the trolley is monitored in real time to ensure smooth operation of the production line.

[0202] [Embodiment of the vision-based trolley wheel positioning and speed detection device of the present application]

[0203] like Figure 4As shown, the embodiment of the present application provides a vision-based trolley wheel positioning and speed detection device, which can be used to implement the vision-based trolley wheel positioning and speed detection method in the above embodiment, such as Figure 4 As shown, the vision-based trolley wheel positioning and speed detection device has an image acquisition module 410, a target detection module 420, a linear velocity determination module 430, and an angular velocity determination module 440.

[0204] An image acquisition module, used to acquire a wheel rotation image of the trolley during its travel as a first image through a camera, wherein the camera lens is arranged on a side of the trolley compartment vertically facing the wheel of the trolley;

[0205] An object detection module, used to detect the first image by using a visual object detection algorithm to obtain the position of the wheel;

[0206] The image acquisition module is also used to: acquire the wheel ROI area in the first image as the second image using the minimum circumscribed rectangle of the wheel;

[0207] The target detection module is also used to: detect the second image by a visual target detection algorithm to obtain the positions of all the studs on the wheel;

[0208] The image acquisition module and the target detection module are also used to: repeat the above image acquisition and detection steps after a preset time interval, and detect the positions of the same wheel and all the studs thereon again;

[0209] A linear speed determination module, used to determine the position change of the wheel according to the position of the same wheel in the first images obtained twice, and determine the linear speed of the wheel based on the position change of the wheel and the preset time interval;

[0210] The angular velocity determination module is used to determine the position change of all the studs on the same wheel according to the positions of all the studs on the same wheel in the second images obtained twice, and determine the angular velocity of the wheel rotation based on the position change of all the studs and the preset time interval.

[0211] For details, please refer to the detailed description in the method embodiment, which will not be repeated here.

[0212] [Embodiment of the vision-based trolley wheel positioning and speed detection system of the present application]

[0213] The embodiment of the present application provides a vision-based trolley wheel positioning and speed detection system, comprising: a camera, a trolley and the vision-based trolley wheel positioning and speed detection device described above;

[0214] The camera lenses are arranged vertically on both sides of the carriage facing the wheels, and the images collected by the cameras are provided to the vision-based trolley wheel positioning and speed detection device.

[0215] [Embodiment of the computing device of the present application]

[0216] Figure 5 900 is a schematic structural diagram of a computing device provided in an embodiment of the present application. The computing device can be used as a vision-based trolley wheel positioning and speed detection device to execute the optional embodiments of the above-mentioned vision-based trolley wheel positioning and speed detection method. The computing device can be a terminal or a chip or chip system inside the terminal. Figure 5 As shown, the computing device 900 includes: a processor 910 , a memory 920 , and a communication interface 930 .

[0217] It should be understood that Figure 5 The communication interface 930 in the computing device 900 shown may be used to communicate with other devices, and may specifically include one or more transceiver circuits or interface circuits.

[0218] The processor 910 may be connected to a memory 920. The memory 920 may be used to store the program code and data. Therefore, the memory 920 may be a storage unit inside the processor 910, or an external storage unit independent of the processor 910, or a component including a storage unit inside the processor 910 and an external storage unit independent of the processor 910.

[0219] Optionally, the computing device 900 may further include a bus. The memory 920 and the communication interface 930 may be connected to the processor 910 via the bus. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 A line without an arrow is used to represent the bus, but this does not mean that there is only one bus or one type of bus.

[0220] It should be understood that in the embodiment of the present application, the processor 910 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 910 may adopt one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present application.

[0221] The memory 920 may include a read-only memory and a random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include a nonvolatile random access memory. For example, the processor 910 may also store information on the device type.

[0222] When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform any operation step of the above method and any optional embodiment thereof.

[0223] It should be understood that the computing device 900 according to the embodiment of the present application can correspond to the corresponding subjects in the methods according to the embodiments of the present application, and the above-mentioned and other operations and / or functions of each module in the computing device 900 are respectively for implementing the corresponding processes of each method of the present embodiment. For the sake of brevity, they will not be repeated here.

[0224] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0225] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0229] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0230] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it is used to execute the above method, which includes at least one of the solutions described in the above embodiments.

[0231] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive lists) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.

[0232] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0233] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0234] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0235] In addition, the words "first, second, third, etc." or module A, module B, module C and other similar terms in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0236] In the above description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed accordingly. If permitted, the order of the steps can be interchanged or they can be executed simultaneously.

[0237] The term "comprising" as used in the description and claims should not be interpreted as being limited to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the features, integers, steps or components mentioned, but does not exclude the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "a device comprising means A and B" should not be limited to a device consisting of components A and B only.

[0238] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics can be combined in any appropriate manner, as would be apparent to one of ordinary skill in the art from this disclosure.

[0239] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, all of which belong to the scope of protection of the present application.

Claims

1. A vision-based trolley wheel positioning and speed detection method, characterized in that: include: The camera is used to obtain a wheel rotation image of the trolley during its travel as a first image, wherein the camera lens is arranged on the side of the trolley compartment vertically facing the wheel of the trolley; Detecting the first image using a visual target detection algorithm to obtain the position of the wheel; The wheel ROI region in the first image is obtained as the second image using the minimum circumscribed rectangle of the wheel; Detecting the second image using a visual target detection algorithm to obtain positions of all studs on the wheel; Repeat the above image acquisition and detection steps after a preset time interval to detect the same wheel and the positions of all the studs thereon again; Determine the position change of the wheel according to the position of the same wheel in the first images obtained twice, and determine the linear speed of the wheel based on the position change of the wheel and the preset time interval; According to the positions of all the studs on the same wheel in the second images obtained twice, the position changes of all the studs are determined, and the angular velocity of the wheel rotation is determined based on the position changes of all the studs and the preset time interval.

2. The method according to claim 1, characterized in that The image is detected by the visual target detection algorithm to obtain the location of the target, including: The image is detected by a visual target detection algorithm to obtain the position information of the minimum circumscribed rectangle of the target; The center position information of the target is obtained based on the position information of the minimum circumscribed rectangle of the target; The image is the first image or the second image, and the target is a wheel or a stud.

3. The method according to claim 2, characterized in that The minimum bounding rectangle is determined as follows: P=(X lt ,Y lt ,Width,Height); Among them, X lt , Y lt is the coordinate of the upper left corner of the minimum bounding rectangle of the wheel or stud, Width and Height are the width and height of the bounding rectangle; P is the position information of the minimum bounding rectangle; The position of the wheel or the stud is obtained based on the position information of the minimum circumscribed rectangle in the following manner: Wherein, C is the center position information of the wheel or stud.

4. The method according to claim 3, characterized in that The position change of the wheel is determined according to the position of the same wheel in the first image obtained twice according to the following formula: ΔX=(d_pix×ΔP)×z / focla Wherein, z is the distance between the camera and the center of the wheel, focal is the focal length of the camera lens, ΔX is the actual distance traveled by the wheel, d_pix is ​​the pixel size of the camera, ΔP is the lateral absolute position change of the center position of the wheel in the pixel coordinate system in the first image obtained twice, and ΔX is the position change of the wheel; The linear speed of the wheel is determined based on the position change of the wheel and the preset time interval according to the following formula: V l =ΔX / t Among them, V l is the linear speed of the wheel, and t is the preset time interval.

5. The method according to claim 2 or 4, characterized in that Determining the position change of all the studs according to the positions of all the studs on the same wheel in the second images obtained twice, and determining the angular velocity of the wheel rotation based on the position change of all the studs and the preset time interval, including: Use the least square method to perform circle fitting on the positions of all the nails detected each time to obtain the center position information of the nails; Calculating the angle change of the same nail in the second images obtained twice according to the center position information of the nail and the center position information of all the nails; The angular velocity of the wheel rotation is determined based on the angle change of the same stud in the second images obtained twice and the preset time interval.

6. The method according to claim 5, characterized in that According to the following formula, the least square method is used to perform circle fitting on the positions of all the studs detected each time to obtain the center position information of the studs: f=∑((x i -x c ) 2 +(y i -y c ) 2 -R 2 ) 2 Where f is the residual sum of squares between all detected nail positions and the fitted circle, x i and i is the center coordinate of the nail; x c and c are the coordinates of the center of the fitted circle; R is the radius.

7. The method according to claim 5, characterized in that Before calculating the angle change of the same nail in the second image obtained twice, it also includes: Taking the positions of all the nails detected for the first time as a reference, the Euclidean distances with the positions of all the nails obtained for the second time are calculated respectively, and the group with the closest Euclidean distance is regarded as the same nail.

8. The method according to claim 7, characterized in that Based on the positions of all the nails detected for the first time, the Euclidean distances with the positions of all the nails obtained for the second time are calculated respectively, and the group with the closest Euclidean distance is regarded as the same nail, including: For each element a i ∈A, calculate a i With each element b in B j The Euclidean distance of: Find the number of cells in B that use d(a i , b j )The smallest element: Among them, A represents the location information set of all the nails detected for the first time, a i It is represented by the elements of A; B represents the location information set of all the studs detected for the first time, b j represents the elements of B; d represents the Euclidean distance between elements; b closest It is expressed as the minimum Euclidean distance of an element in B that is considered the same as an element in A.

9. The method according to claim 6, characterized in that Calculating the angle change of the same nail in the second images obtained twice according to the center position information of the nail and the center position information of all the nails, including: The first vector v1 and the second vector v2 from the same nail to the center of the circle are calculated twice: v1=(x i 1-x c 1,y i 1-y c 1) v2=(x i 2-x c 2,y i 2nd year c 2) Calculate the dot product of the first vector v1 and the second vector v2: v1·v2=(x i 1-x c 1)×(x i 2-x c 2)+(y i 1-y c 1)×(y i 2-y c 2) Calculate the magnitude of the first vector v1 and the second vector v2: Calculate the cosine of the angle as the ratio of the dot product of the first vector v1 and the second vector v2 to the product of the magnitude of the first vector and the magnitude of the second vector: Calculate the angle θ and convert it to radians: Among them, x i 1.y i 1 is the center position coordinate of any nail obtained for the first time; x i 2.y i 2 is the center coordinate of the same nail obtained for the second time; x c 1 and y c 1 is the coordinate of the center of the fitting circle obtained for the first time, x c 2 and y c 2 is the coordinate of the center of the fitting circle obtained for the first time, and the angle θ is the angle change of the same nail in the second image obtained twice.

10. The method according to claim 9, characterized in that The angular velocity of the wheel rotation is determined based on the angle change of the same stud in the second images obtained twice and the preset time interval according to the following formula: Wherein, t is the preset time interval.

Images