Automatic wheel set loading and unloading method and system based on machine vision
Through the automatic loading and unloading method based on machine vision, using grayscale image and filtering processing technology, the problems of low safety and efficiency during the loading and unloading of wheel pairs are solved, and safe and efficient automatic loading and unloading are achieved.
Patent Information
- Application Number
- CN202411203385.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The prior art is difficult to ensure safety during the wheel loading and unloading process, and requires manual assisted positioning, which has problems of high labor intensity and low efficiency.
Using an automatic loading and unloading method based on machine vision, the grayscale image of the wheel pair is obtained, the characteristic value and matching degree value of the pixel points are determined, and the filtering process and edge detection are performed to realize the automatic loading and unloading of the wheel pair and reduce manual participation.
The safety and efficiency of the wheelset loading and unloading process is improved, the intensity of manual labor is reduced, and the threat of wheelset to assist positioning personnel in the sports state is avoided, thus achieving more accurate loading and unloading operations.
Smart Images

Figure CN118762083B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image data processing, and in particular to a method and system for automatic loading and unloading of wheelsets based on machine vision. Background Art
[0002] Wheel sets are the core components that ensure the operation of rail vehicles. Wheel sets are commonly used in rail vehicles such as subways, trains and light rails. Wheel sets include two wheel discs and an axle. The axle connects the two wheel discs located at both ends. The wheelset can bear the weight of the rail vehicle and transfer the weight from the vehicle to the track.
[0003] In order to realize the loading and unloading of wheelsets, in the relevant technology, for example, in the Chinese patent application document with publication number CN114148373A, a wheelset installation device is provided, which is suitable for the installation of wheelsets of trains between a first track and a second track with different track gauges, the first track is suitable for the first wheelset, the second track is suitable for the second wheelset, there is a distance between the first track and the second track, and the device comprises: an installation track, the installation track comprises a second track portion arranged along the running direction of the train, the height of the second track portion is lower than the height of the first track, and the height of the middle part of the second track portion is higher than the height of both sides of the second track portion; an installation vehicle for carrying the second wheelset, the installation vehicle moves synchronously with the train on the second track portion of the installation track; wherein: when the train leaves the first track, the first wheelset on the train is disassembled; the installation vehicle moves synchronously with the train from which the first wheelset is disassembled on the second track portion of the installation track; the installation vehicle assembles the second wheelset to the train from which the first wheelset is disassembled.
[0004] However, when the related technology is used to install or disassemble the wheelset, manual auxiliary positioning of the wheelset is required, and the wheelset in motion may pose a threat to the manual auxiliary positioning, making it difficult to ensure the safety of the wheelset loading and unloading process. Summary of the invention
[0005] In order to overcome the problem in the related art that it is difficult to ensure the safety of the wheelset loading and unloading process, the present application provides a method and system for automatic wheelset loading and unloading based on machine vision.
[0006] According to a first aspect of an embodiment of the present application, there is provided a method for automatic loading and unloading of wheelsets based on machine vision, comprising: obtaining a first grayscale image of a first end of a target wheelset, and determining a characteristic value of the first pixel according to the grayscale value of a pixel within a preset range of the first pixel in the first grayscale image; obtaining a second grayscale image of a second end of the target wheelset, determining a second pixel corresponding to the same position point as the first pixel from the second grayscale image, and determining the characteristic value of the second pixel according to the grayscale value of a pixel within a preset range of the second pixel; determining a matching degree value between the first pixel and the second pixel according to a first difference value in the characteristic value between the first pixel and the second pixel; filtering the first pixel whose matching degree value in the first grayscale image is less than a preset threshold to obtain a first target image, and filtering the second pixel whose matching degree value in the second grayscale image is less than the preset threshold to obtain a second target image; loading and unloading the target wheelset according to the first target image and the second target image.
[0007] In this way, the loading and unloading process of the wheelset in the embodiment of the present application is carried out based on the first target image and the second target image, which can reduce manual participation in the loading and unloading process of the wheelset. Therefore, the threat posed by the wheelset in motion to the manual personnel performing auxiliary positioning can be avoided, thereby ensuring the safety of the loading and unloading process of the wheelset.
[0008] At the same time, by obtaining the characteristic value of the first pixel point in the first grayscale image of the first end of the target wheelset, and determining the characteristic value of the second pixel point corresponding to the first pixel point in the second grayscale image of the second end of the target wheelset, so as to determine the matching degree value between the first pixel point and the second pixel point, the first grayscale image and the second grayscale image can be processed to obtain the first target image and the second target image, and the noise that may exist in the first grayscale image and the second grayscale image can be avoided. Therefore, the loading and unloading of the target wheelset can be performed through the first target image and the second target image, so the loading and unloading of the wheelset can be achieved more accurately.
[0009] Furthermore, compared with manually monitoring the loading and unloading process of the wheelset and performing auxiliary positioning, the loading and unloading efficiency of the wheelset can be improved and the labor intensity of manual labor can be reduced.
[0010] Optionally, determining the matching degree value between the first pixel point and the second pixel point based on the first difference value between the first pixel point and the second pixel point in the eigenvalue includes: determining the first variance of the eigenvalue of all pixels within a preset range of the first pixel point, and determining the second variance of the eigenvalue of all pixels within a preset range of the second pixel point, so as to determine the second difference value between the first variance and the second variance; determining the first inverse of the first difference value, and determining the second inverse of the second difference value, so as to multiply the first inverse by the second inverse to obtain the matching degree value.
[0011] In this way, the second variance of the characteristic values of all pixels within a preset range of the second pixel is determined, and the second difference value between the first variance and the second variance is determined; the first inverse of the first difference value is determined, and the second inverse of the second difference value is determined, so as to multiply the first inverse and the second inverse to obtain the matching degree value, and the matching degree between the first pixel and the second pixel can be effectively represented by the obtained matching degree value.
[0012] Optionally, the first difference value between the first pixel point and the second pixel point in the characteristic value is determined by: , wherein M1 is the first difference value between the first pixel and the second pixel in the characteristic value, A1 is the characteristic value of the first pixel, A2 is the characteristic value of the second pixel, and T is a preset value greater than 0.
[0013] In this way, the first difference value determined in the embodiment of the present application can better characterize the degree of difference in characteristic values between the first pixel and the second pixel, and the first difference value can also better characterize the degree of difference in grayscale information between the first pixel and the second pixel.
[0014] Optionally, the second difference value between the first variance and the second variance is determined in the following manner: , wherein M2 is the second difference value between the first variance and the second variance, D1 is the first variance, D2 is the second variance, and T is a preset value greater than 0.
[0015] In this way, the second difference value determined in the embodiment of the present application can better characterize the difference between the first variance and the second variance, thereby better characterizing the difference between the first pixel and the second pixel in grayscale information.
[0016] Optionally, the characteristic value of the first pixel is determined based on the grayscale mean of all pixels within a preset range of the first pixel in the first grayscale image; the characteristic value of the second pixel is determined based on the grayscale mean of all pixels within a preset range of the second pixel in the second grayscale image.
[0017] Optionally, the target wheelset is loaded and unloaded according to the first target image and the second target image, including: performing edge detection on the first target image to obtain a first edge image; performing edge detection on the second target image to obtain a second edge image; determining a first position of the wheel center of the target wheelset according to the first edge image and the second edge image, and loading and unloading the target wheelset according to the first position.
[0018] In this way, since the first target image is the image after denoising the first grayscale image, the first edge image corresponding to the first target image can better reflect the edge information of the first end of the wheelset; the second target image is the image after denoising the second grayscale image, and the second edge image corresponding to the second target image can better reflect the edge information of the second end of the wheelset. Therefore, based on the first edge image and the second edge image, the first position of the wheel center of the target wheelset can be better determined, and the target wheelset can be loaded and unloaded according to the first position.
[0019] Optionally, an inertial measurement unit is provided on the wheelset, and the method further includes: receiving inertial measurement data collected by the inertial measurement unit, and obtaining an initial position of the wheelset moving in the wheelset parking area; determining position information of the wheelset in the wheelset parking area at a current moment according to the initial position and the inertial measurement data; and generating a wheelset position distribution map corresponding to the parking area at a current moment according to different position information of different wheelsets in the parking area at a current moment.
[0020] In this way, based on the different position information of different wheelsets in the parking area at the current moment, a wheelset position distribution map corresponding to the parking area at the current moment can be generated, so that the wheelset management personnel can know the position distribution of the wheelsets in the parking area more intuitively.
[0021] Optionally, the method also includes: obtaining first point cloud data of a target area where the first wheelset to be loaded is located, and obtaining a third target image of the target area; matching the first point cloud data and the third target image to determine the second position of the first wheelset to be loaded in space; sending a first control instruction to a wheelset moving mechanism so that the wheelset moving mechanism moves the first wheelset to be loaded from the second position to the target position, wherein the target position is a position where a first target vehicle for transporting the first wheelset to be loaded is parked.
[0022] Optionally, the method also includes: when unloading the wheelset loaded in the second target vehicle, sending a third control instruction to the wheelset moving mechanism to move the wheelset moving mechanism to above the loading area of the second target vehicle, the second target vehicle being a vehicle loaded with the wheelset to be unloaded; acquiring a fourth target image of the loading area, and acquiring second point cloud data of the loading area; determining, from the loading area, based on the second point cloud data and the fourth target image, the second wheelset with the smallest vertical distance from the wheelset moving mechanism; and sending a fourth control instruction to the wheelset moving mechanism to enable the wheelset moving mechanism to unload the second wheelset from the second target vehicle.
[0023] According to a second aspect of an embodiment of the present application, there is provided a wheelset automatic loading and unloading system based on machine vision, comprising: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the steps of the wheelset automatic loading and unloading method based on machine vision provided in the first aspect of the present application are implemented.
[0024] The technical solution provided by the embodiments of the present application may include the following beneficial effects: the loading and unloading process of the wheelset is carried out based on the first target image and the second target image, which can reduce manual participation in the loading and unloading process of the wheelset, thereby avoiding the threat posed by the wheelset in motion to the manual personnel performing auxiliary positioning, thereby ensuring the safety of the loading and unloading process of the wheelset.
[0025] At the same time, by obtaining the characteristic value of the first pixel point in the first grayscale image of the first end of the target wheelset, and determining the characteristic value of the second pixel point corresponding to the first pixel point in the second grayscale image of the second end of the target wheelset, so as to determine the matching degree value between the first pixel point and the second pixel point, the first grayscale image and the second grayscale image can be processed to obtain the first target image and the second target image, and the noise that may exist in the first grayscale image and the second grayscale image can be avoided. Therefore, the loading and unloading of the target wheelset can be performed through the first target image and the second target image, so the loading and unloading of the wheelset can be achieved more accurately.
[0026] Furthermore, compared with manually monitoring the loading and unloading process of the wheelset and performing auxiliary positioning, the loading and unloading efficiency of the wheelset can be improved and the labor intensity of manual labor can be reduced.
[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0029] Figure 1 is a flow chart of a method for automatic loading and unloading of wheelsets based on machine vision according to an exemplary embodiment;
[0030] Figure 2 is a wheelset position distribution diagram shown according to an exemplary embodiment;
[0031] Figure 3 It is a schematic structural diagram of a wheelset automatic loading and unloading system based on machine vision according to an exemplary embodiment. DETAILED DESCRIPTION
[0032] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.
[0033] First, the application scenario of the embodiment of the present application is briefly introduced. In the application scenario of the present application, in order to realize the disassembly of the wheelset, the image acquisition equipment can be used to detect whether the installation position or installation direction of the wheelset is installed. However, when the installation position or installation direction of the wheelset is already wrong, the cost of correcting the installation position or installation direction of the wheelset is high. The related technology fails to realize the supervision of the loading and unloading process of the wheelset, which may affect the safety of the rail vehicle.
[0034] In response to the above technical problems, an embodiment of the present application provides a method and system for automatic loading and unloading of wheelsets based on machine vision. The loading and unloading process of the wheelset is performed according to the first target image and the second target image, which can reduce manual participation in the loading and unloading process of the wheelset. Therefore, the threat posed by the wheelset in motion to the manual personnel performing auxiliary positioning can be avoided, thereby ensuring the safety of the loading and unloading process of the wheelset.
[0035] At the same time, by obtaining the characteristic value of the first pixel point in the first grayscale image of the first end of the target wheelset, and determining the characteristic value of the second pixel point corresponding to the first pixel point in the second grayscale image of the second end of the target wheelset, so as to determine the matching degree value between the first pixel point and the second pixel point, the first grayscale image and the second grayscale image can be processed to obtain the first target image and the second target image, and the noise that may exist in the first grayscale image and the second grayscale image can be avoided. Therefore, the loading and unloading of the target wheelset can be performed through the first target image and the second target image, so the loading and unloading of the wheelset can be achieved more accurately.
[0036] Furthermore, compared with manually monitoring the loading and unloading process of the wheelset and performing auxiliary positioning, the loading and unloading efficiency of the wheelset can be improved and the labor intensity of manual labor can be reduced.
[0037] Figure 1 is a flowchart of a method for automatic loading and unloading of wheelsets based on machine vision according to an exemplary embodiment, which can be applied to a loading and unloading monitoring terminal of a wheelset, such as Figure 1 As shown, the method includes the following steps.
[0038] In step S101, a first grayscale image of a first end of a target wheelset is acquired, and a feature value of a first pixel is determined according to grayscale values of pixels within a preset range of the first pixel in the first grayscale image.
[0039] For example, a first image acquisition device and a second image acquisition device may be provided, and the first image acquisition device and the second image acquisition device may be located at different ends of the target wheelset, respectively, and the positions of the image acquisition devices provided at both ends of the target wheelset may be matched in advance so as to determine the same wheelset corresponding to the images acquired by the first image acquisition device and the second image acquisition device.
[0040] When it is determined that the images captured by the first image acquisition device and the second image acquisition device correspond to the same wheel pair, it is also possible to determine that the pixel pairs of the same position points of the wheelset corresponding to the images captured by the first image acquisition device and the second image acquisition device are located in the images captured by the first image acquisition device and the second image acquisition device, respectively.
[0041] The first grayscale image can be acquired by a first image acquisition device disposed at the first end of the wheelset, or by graying a visible light image acquired by the first image acquisition device disposed at the first end of the wheelset.
[0042] The characteristic value of the first pixel can be determined according to the grayscale value of the pixel within a preset range of the first pixel in the first grayscale image. The preset range can be a range centered on the first pixel. For example, the preset range can be a pre-set range of a size such as 5×5 or 7×7. The characteristic value of the first pixel can be determined according to the grayscale mean of all pixels within the preset range of the first pixel in the first grayscale image. For example, the characteristic value of the first pixel can be equal to the grayscale mean of all pixels within the preset range of the first pixel in the first grayscale image.
[0043] In step S102, a second grayscale image of the second end of the target wheel pair is obtained, a second pixel point corresponding to the same position point as the first pixel point is determined from the second grayscale image, and a characteristic value of the second pixel point is determined based on the grayscale value of the pixel point within a preset range of the second pixel point.
[0044] The second grayscale image of the second end of the target wheelset can be acquired by a second image acquisition device disposed at the second end of the target wheelset. The second end and the first end are different ends of the target wheelset. The second grayscale image may include a wheel disk located at the second end of the target wheelset.
[0045] When noise exists in the image of the wheel disk of the wheelset, it may be difficult to locate the center of the wheel axle of the target wheelset. To determine whether noise exists in the second image, a second pixel point corresponding to the same position point as the first pixel point may be determined from the second grayscale image.
[0046] For example, when the center of the wheel disk of the wheelset is the origin of the plane coordinate system, if the first pixel point in the first image is located at position (1, 2), the pixel point at position (1, 2) in the second image can be used as the second pixel point corresponding to the first pixel point.
[0047] The characteristic value of the second pixel can be determined based on the grayscale value of the pixel within a preset range of the second pixel. The characteristic value of the second pixel is determined based on the grayscale mean of all pixels within the preset range of the second pixel in the second grayscale image. For example, the characteristic value of the second pixel can be equal to the grayscale mean of all pixels within the preset range of the second pixel in the second grayscale image. The preset range can be a pre-set range of size such as 5×5 or 7×7.
[0048] In step S103, a matching degree value between the first pixel point and the second pixel point is determined according to a first difference value between the first pixel point and the second pixel point in feature values.
[0049] When there is no noise in the first image and the second image, the first pixel and the second pixel usually exhibit the same or similar grayscale values; when there is noise in the first image or the second image, the grayscale value of the first pixel may be affected by the noise, or the grayscale value of the second pixel may be affected by the noise, such that the difference between the grayscale information of the first pixel and the second pixel becomes larger.
[0050] The eigenvalue of the first pixel can characterize the grayscale information of the first pixel, and the eigenvalue of the second pixel can characterize the grayscale information of the second pixel. Therefore, the first difference value in the eigenvalues between the first pixel and the second pixel can characterize the degree of difference in grayscale information between the first pixel and the second pixel, so as to determine whether there are pixels whose grayscale values are affected by noise in the first image or the second image.
[0051] In one embodiment, the first difference value between the first pixel point and the second pixel point in the characteristic value is determined by: , wherein M1 is the first difference value between the first pixel and the second pixel in the characteristic value, A1 is the characteristic value of the first pixel, A2 is the characteristic value of the second pixel, and T is a preset value greater than 0.
[0052] When the eigenvalue of the first pixel is equal to the eigenvalue of the second pixel, the values of the numerator and denominator of M1 are equal, so that the value of the first difference value is 1; and when the eigenvalue of the first pixel is not equal to the eigenvalue of the second pixel, the value of the numerator of M1 is greater than the value of the denominator, so that the value of the first difference value is greater than 1; and the greater the difference between the eigenvalue of the first pixel and the second pixel in the eigenvalue, the greater the value of M1. The existence of a preset value T greater than 0 can effectively avoid the situation where the denominator is 0, and the preset value can be, for example, between 0.1 and 0.5.
[0053] In this way, the first difference value determined in the embodiment of the present application can better characterize the degree of difference in characteristic values between the first pixel and the second pixel, and the first difference value can also better characterize the degree of difference in grayscale information between the first pixel and the second pixel.
[0054] The larger the first difference value in the eigenvalue between the first pixel point and the second pixel point, the lower the matching degree between the first pixel point and the second pixel point, and the smaller the matching degree value between the first pixel point and the second pixel point; conversely, the larger the first difference value in the eigenvalue between the first pixel point and the second pixel point, the higher the matching degree between the first pixel point and the second pixel point, and the larger the matching degree value between the first pixel point and the second pixel point; therefore, based on the first difference value in the eigenvalue between the first pixel point and the second pixel point, the matching degree value between the first pixel point and the second pixel point can be determined, so as to characterize the matching degree between the first pixel point and the second pixel point by the matching degree value.
[0055] In step S104, the first pixel points in the first grayscale image whose matching degree values are less than the preset threshold are filtered to obtain the first target image, and the second pixel points in the second grayscale image whose matching degree values are less than the preset threshold are filtered to obtain the second target image.
[0056] Filtering is an important means in signal processing and image processing. It can effectively remove noise that may exist in the image and retain valuable texture information in the image. Pixel points can be filtered by using a target filter. The target filter can be one or more of the following: low-pass filter, high-pass filter, band-pass filter, mean filter, median filter, Gaussian filter, bilateral filter, and Kalman filter.
[0057] For the first pixel points in the first grayscale image whose matching degree values are less than a preset threshold, filtering processing can be performed to avoid the influence of noise on the grayscale values of these pixels; and for the first pixel points in the first grayscale image whose matching degree values are less than or equal to the preset threshold, the texture information of the pixel points can be retained; wherein the preset threshold can be determined based on the value range of the matching degree value, for example, when the matching degree value is positive and the maximum value is 1, the preset threshold can be between 0.1 and 0.3.
[0058] For the second pixel points in the second grayscale image whose matching degree values are less than a preset threshold, filtering processing can be performed to avoid the influence of noise on the grayscale values of these pixel points; and for the second pixel points in the second grayscale image whose matching degree values are less than or equal to the preset threshold, the texture information of the pixel points can be retained.
[0059] Furthermore, the first pixel point is located in the first grayscale image, the second pixel point is located in the second grayscale image, and the first pixel point and the second pixel point correspond to the same position of the roulette wheels on both sides. The probability that the grayscale values of the first pixel point and the second pixel point are simultaneously affected by noise is relatively small, and the matching degree value between the first pixel point and the second pixel point is less than a preset threshold, indicating that either the first pixel point or the second pixel point has a greater probability of being affected by noise, and the noise that may exist in the image can be effectively eliminated by filtering the pixel points.
[0060] In step S105, the target wheelset is loaded and unloaded according to the first target image and the second target image.
[0061] The image coordinate system and the space coordinate system of the first target image can be calibrated in advance to obtain a first transformation relationship, and the first transformation relationship is used to characterize the transformation relationship between the image coordinate system and the space coordinate system of the first target image; the image coordinate system and the space coordinate system of the second target image can be calibrated in advance to obtain a second transformation relationship, and the second transformation relationship is used to characterize the transformation relationship between the image coordinate system and the space coordinate system of the second target image.
[0062] According to the first conversion relationship and the coordinates of the first end of the wheelset in the first target image, the position of the first end of the wheelset in the spatial coordinate system can be determined; according to the second conversion relationship and the coordinates of the second end of the wheelset in the second target image, the position of the second end of the wheelset in the spatial coordinate system can be determined, so as to load the target wheelset onto the vehicle, or unload the target wheelset from the vehicle loaded with the wheelset.
[0063] In one embodiment, a target wheelset is loaded and unloaded according to a first target image and a second target image, including: performing edge detection on the first target image to obtain a first edge image; performing edge detection on the second target image to obtain a second edge image; determining a first position of a wheel center of the target wheelset according to the first edge image and the second edge image, and loading and unloading the target wheelset according to the first position.
[0064] Edge detection is an important method in image processing and computer vision. It can effectively determine the pixels located at the edge of the image. Edge detection can be achieved through the Sobel operator, Prewitt operator, Roberts operator, Scharr operator, Laplacian operator and Canny edge detector. Different edge detection operators correspond to different convolution kernels. By convolving the grayscale image with the convolution kernel, the edge image corresponding to the grayscale image can be obtained.
[0065] A first edge image is obtained by performing edge detection on the first target image, and the first edge image can effectively reflect the edge information of the first end of the wheelset in the first target image; a second edge image is obtained by performing edge detection on the second target image, and the second edge image can effectively reflect the edge information of the second end of the wheelset in the second target image.
[0066] In this way, since the first target image is the image after denoising the first grayscale image, the first edge image corresponding to the first target image can better reflect the edge information of the first end of the wheelset; the second target image is the image after denoising the second grayscale image, and the second edge image corresponding to the second target image can better reflect the edge information of the second end of the wheelset. Therefore, based on the first edge image and the second edge image, the first position of the wheel center of the target wheelset can be better determined, and the target wheelset can be loaded and unloaded according to the first position.
[0067] Furthermore, compared with directly determining the position of the roulette wheel center based on the first target image and the second target image, the edge image can reduce the amount of processing required for calculation while effectively retaining the edge information of the roulette wheel of the wheelset, which is beneficial to reducing the power consumption of the device.
[0068] The roulette wheel center refers to the center of the roulette wheel of the wheelset. For example, since the shape of the roulette wheel is circular, an edge line located at the center of the first edge image can be determined, and the center of the edge line is used as the roulette wheel center of the wheelset.
[0069] Through the automatic loading and unloading method of wheelsets based on machine vision provided in the embodiments of the present application, the loading and unloading process of the wheelsets is performed based on the first target image and the second target image, which can reduce manual participation in the loading and unloading process of the wheelsets. Therefore, the threat posed by the wheelsets in motion to the manual personnel performing auxiliary positioning can be avoided, thereby ensuring the safety of the loading and unloading process of the wheelsets.
[0070] At the same time, by obtaining the characteristic value of the first pixel point in the first grayscale image of the first end of the target wheelset, and determining the characteristic value of the second pixel point corresponding to the first pixel point in the second grayscale image of the second end of the target wheelset, so as to determine the matching degree value between the first pixel point and the second pixel point, the first grayscale image and the second grayscale image can be processed to obtain the first target image and the second target image, and the noise that may exist in the first grayscale image and the second grayscale image can be avoided. Therefore, the loading and unloading of the target wheelset can be performed through the first target image and the second target image, so the loading and unloading of the wheelset can be achieved more accurately.
[0071] Furthermore, compared with manually monitoring the loading and unloading process of the wheelset and performing auxiliary positioning, the loading and unloading efficiency of the wheelset can be improved and the labor intensity of manual labor can be reduced.
[0072] In one embodiment, a matching degree value between a first pixel point and a second pixel point is determined based on a first difference value between the first pixel point and the second pixel point in eigenvalues, including: determining a first variance of the eigenvalues of all pixels within a preset range of the first pixel point, and determining a second variance of the eigenvalues of all pixels within a preset range of the second pixel point, so as to determine a second difference value between the first variance and the second variance; determining a first inverse of the first difference value, and determining a second inverse of the second difference value, so as to multiply the first inverse by the second inverse to obtain the matching degree value.
[0073] The first variance of the eigenvalues of all pixels within the preset range of the first pixel point can characterize the degree of difference in the grayscale information of the pixels within the preset range of the first pixel point, and can also reflect the texture information within the preset range of the first pixel point; the second variance of the eigenvalues of all pixels within the preset range of the second pixel point can characterize the degree of difference in the grayscale information of the pixels within the preset range of the second pixel point, and can also reflect the texture information within the preset range of the second pixel point.
[0074] When the first pixel is affected by noise, the texture information within a preset range of the first pixel will be destroyed; when the second pixel is affected by noise, the texture information within a preset range of the second pixel will be destroyed; when any one of the first pixel and the second pixel is affected by noise, the second difference value between the first variance and the second variance will become larger, and the probability that the first pixel and the second pixel are affected by noise at the same time is small. Therefore, the second difference value between the first variance and the second variance can better characterize the degree of difference between the first pixel and the second pixel.
[0075] In this way, the second variance of the characteristic values of all pixels within a preset range of the second pixel is determined, and the second difference value between the first variance and the second variance is determined; the first inverse of the first difference value is determined, and the second inverse of the second difference value is determined, so as to multiply the first inverse and the second inverse to obtain the matching degree value, and the matching degree between the first pixel and the second pixel can be effectively represented by the obtained matching degree value.
[0076] In one embodiment, the second difference between the first variance and the second variance is determined by: , wherein M2 is the second difference value between the first variance and the second variance, D1 is the first variance, D2 is the second variance, and T is a preset value greater than 0.
[0077] When the first variance is equal to the second variance, the values of the numerator and denominator of M2 are equal, so that the value of the second difference value is 1; when the first variance is not equal to the second variance, the value of the numerator of M2 is greater than the value of the denominator, so that the value of the second difference value is greater than 1; and, the greater the difference between the first variance and the second variance, the greater the value of M2; the existence of a preset value T greater than 0 can effectively avoid the situation where the denominator is 0, and the preset value can be, for example, between 0.1 and 0.5.
[0078] In this way, the second difference value determined in the embodiment of the present application can better characterize the difference between the first variance and the second variance, thereby better characterizing the difference between the first pixel and the second pixel in grayscale information.
[0079] For example, the matching degree value between the first pixel point and the second pixel point can be determined in the following manner: , where P is the matching degree value between the first pixel and the second pixel, A1 is the characteristic value of the first pixel, A2 is the characteristic value of the second pixel, T is a preset value greater than 0, D1 is the first variance, and D2 is the second variance.
[0080] In order to facilitate the comparison of the matching degree value with the preset threshold, the matching degree value can also be normalized so that the matching degree value is within the range of (0, 1); wherein, the normalization of the matching degree value can be achieved by using minimum-maximum normalization and logarithmic normalization.
[0081] In one embodiment, the wheelset is provided with an inertial measurement unit, and can also receive inertial measurement data collected by the inertial measurement unit, and obtain the initial position of the wheelset moving in the wheelset parking area; determine the position information of the wheelset in the wheelset parking area at the current moment according to the initial position and the inertial measurement data; and generate a wheelset position distribution map corresponding to the parking area at the current moment according to the different position information of different wheelsets in the parking area at the current moment.
[0082] For example, the inertial measurement unit can be wirelessly connected to the terminal device, and through the established wireless connection, the inertial measurement unit can send the collected inertial measurement data to the terminal device. The inertial measurement data may include acceleration information and angular velocity information when the wheelset moves; the number of inertial measurement units set on the same wheelset can be one or more.
[0083] Among them, the acceleration information may include acceleration information of the inertial measurement unit of the wheelset in different directions; the angular velocity information may include angular velocity information of the inertial measurement unit of the wheelset when rotating around different axes, such as angular velocity information when rotating in the vertical direction.
[0084] By setting an inertial measurement unit on the wheelset, the movement process of the wheelset can be monitored. Combined with the initial position of the wheelset in the parking area, the specific position of the wheelset in the parking area can be determined. This not only makes it easier for the wheelset management personnel to determine the position of the wheelset and the number of wheelsets in the parking area, but also makes it easier for the wheelset moving mechanism to move to the position of the wheelset to realize the movement of the wheelset.
[0085] Tracks for parking wheelsets may be provided in the parking area, and the intervals between the tracks may be determined according to the distance between the two wheel discs of the wheelset. When the same parking area is used to park wheelsets with different wheelbases, tracks with different intervals may also be provided. In order to accommodate wheelsets with different wheelbases, the ground where the tracks are located may also be provided with an adjustment mechanism, and the adjustment mechanism is used to adjust the position of the tracks so as to adjust the distance between the two tracks.
[0086] According to the different position information of different wheelsets in the parking area at the current moment, a wheelset position distribution map corresponding to the parking area at the current moment can be generated, so that the wheelset management personnel can know the position distribution of the wheelsets in the parking area more intuitively.
[0087] Figure 2 is an exemplary wheelset position distribution diagram of an embodiment of the present application, such as Figure 2 As shown, there are wheel sets parked in all parking positions in the first row and the second row of the parking area, no wheel sets are parked in the third row of the fourth row, and no wheel sets are parked in all parking positions in the fourth row.
[0088] In order to facilitate the managers of the wheelset to understand the information of the parking area, the wheelbase information and wheel size information of the parked wheelset can also be displayed ( Figure 2 (not shown in the figure), the wheelbase information includes the distance between the two wheel discs of the wheelset; or, in order to make the interface more concise, the wheelbase information and wheel disc size information of the wheelset corresponding to the target touch point can also be displayed in response to the triggering operation of the target touch point on the screen of the terminal device.
[0089] In one embodiment, first point cloud data of a target area where a first wheelset to be loaded is located is obtained, and a third target image of the target area is obtained; the first point cloud data and the third target image are matched to determine a second position of the first wheelset to be loaded in space; and a first control instruction is sent to a wheelset moving mechanism so that the wheelset moving mechanism moves the first wheelset to be loaded from the second position to a target position, wherein the target position is a position where a first target vehicle for transporting the first wheelset to be loaded is parked.
[0090] The target area may be an area where the wheelsets to be loaded are parked, and the wheelset moving mechanism may be used to move the wheelsets from the target area to the first target vehicle so that the first target vehicle can transport the wheelsets.
[0091] The wheelset moving mechanism can realize the movement of the wheelset on the horizontal plane and in the vertical direction. For example, the wheelset moving mechanism may include a first driving component, a second driving component and a third driving component, wherein the first driving component is used to drive the wheelset to move in the first horizontal direction, the second driving component can be used to drive the wheelset to move in the second horizontal direction, and the third driving component can drive the wheelset to move in the vertical direction. The first horizontal direction and the second horizontal direction are two directions perpendicular to each other on the horizontal plane.
[0092] The first driving component, the second driving component and the third driving component can be moved by a gear rack structure; one end of the third driving component is connected to the second driving component, and the other end of the third driving component is connected to a grasping component, which can be, for example, a robotic arm or a clamp, and the grasping component can be rotated to a target angle along the vertical direction according to the received control command.
[0093] The gripping component can grasp and release the wheelset through hydraulic transmission. The gripping component can be provided with an image acquisition device and a point cloud acquisition device so as to realize the positioning of the wheelset to be moved through image data and point cloud data.
[0094] The point cloud data acquisition device can be a laser radar. The point cloud data collected by the point cloud data acquisition device can determine the position information of the wheelset in space; by matching the first point cloud data and the third target image, the point cloud data can be combined with the image data, which can effectively realize the positioning of the wheelset in space and the grasping after positioning.
[0095] The wheelset moving mechanism can be communicatively connected with the terminal device so that the wheelset moving mechanism receives control instructions from the terminal device. The terminal device can send a first control instruction to the wheelset moving mechanism so that the wheelset moving mechanism moves the first wheelset to be loaded from the second position to the target position, and the target position is the position where the first target vehicle for transporting the first wheelset to be loaded is parked.
[0096] In this way, by utilizing the acquired first point cloud data and the third target image, the wheelset moving mechanism can position the first wheelset to be loaded, and a first control instruction is sent to the wheelset moving mechanism so that the wheelset moving mechanism moves the first wheelset to be loaded from the second position to the target position, thereby effectively realizing the loading of the wheelset.
[0097] In one embodiment, when unloading the wheelset loaded in the second target vehicle, a third control instruction is sent to the wheelset moving mechanism to move the wheelset moving mechanism to above the loading area of the second target vehicle, where the second target vehicle is a vehicle loaded with the wheelset to be unloaded; a fourth target image of the loading area is acquired, and second point cloud data of the loading area is acquired; based on the second point cloud data and the fourth target image, a second wheelset with the smallest vertical distance from the wheelset moving mechanism is determined from the loading area; and a fourth control instruction is sent to the wheelset moving mechanism to enable the wheelset moving mechanism to unload the second wheelset from the second target vehicle.
[0098] For example, the fourth target image can be instance segmented to determine each wheelset in the loading area. The vertical distance between the wheelset moving mechanism and each wheelset in the loading area can be determined based on the point cloud data to achieve the positioning of the wheelset in the loading area. The wheelset with the smallest vertical distance from the wheelset moving mechanism is easier to unload, so all the wheelsets in the loading area can be easily unloaded.
[0099] In this way, based on the acquired second point cloud data and the fourth target image, the wheelset moving mechanism can position the wheelset in the loading area of the second target vehicle. Compared with the operator of the wheelset moving mechanism positioning the wheelset under the guidance of a supervisor located near the loading area, the wheelset positioning efficiency can be improved so that the wheelset can be unloaded from the loading area of the second target vehicle.
[0100] Figure 3 FIG. 1 is a schematic diagram showing a structure of a wheel set automatic loading and unloading system based on machine vision according to an exemplary embodiment. Figure 3 The automatic wheelset loading and unloading system based on machine vision includes: a processor 1100 and a memory 1200, wherein the memory 1200 stores computer program instructions, and when the computer program instructions are executed by the processor 1100, all or part of the steps of the automatic wheelset loading and unloading method based on machine vision in the present application are implemented.
[0101] It should be understood that the features of some embodiments of the various applications described herein may be combined with each other, unless specifically stated otherwise.
[0102] Although terms such as "first", "second", and "third" may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Instead, these terms are only used to distinguish one component, part, region, layer, or section from another component, part, region, layer, or section. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section.
[0103] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description herein, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0104] In addition, the word "exemplary" is used herein to mean serving as an example, instance, or diagram. Any aspect or design described herein as "exemplary" is not necessarily to be understood as being advantageous over other aspects or designs. Instead, the use of the word exemplary is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or".
[0105] Likewise, although the present application has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. With particular regard to the various functions performed by the components (e.g., elements, resources, etc.) described above, unless otherwise noted, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if not structurally equivalent to the disclosed structures.
[0106] Additionally, while particular features of the present application may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and advantageous for any given or particular application.
[0107] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application, and the specification and embodiments are only considered as exemplary.
[0108] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.
Claims
1. A method for automatic loading and unloading of wheelsets based on machine vision. It is characterized in that include: Acquire a first grayscale image of a first end of a target wheelset, and determine a characteristic value of the first pixel according to grayscale values of pixels within a preset range of the first pixel in the first grayscale image; Acquire a second grayscale image of the second end of the target wheel pair, determine a second pixel point corresponding to the same position point as the first pixel point from the second grayscale image, and determine a characteristic value of the second pixel point according to the grayscale values of the pixel points within a preset range of the second pixel point; Determining a matching degree value between the first pixel point and the second pixel point according to a first difference value between the first pixel point and the second pixel point in the characteristic value; Performing filtering processing on first pixel points whose matching degree values are less than a preset threshold in the first grayscale image to obtain a first target image, and performing filtering processing on second pixel points whose matching degree values are less than a preset threshold in the second grayscale image to obtain a second target image; The target wheelset is loaded and unloaded according to the first target image and the second target image.
2. The method for automatic wheel set loading and unloading based on machine vision according to claim 1, It is characterized in that The determining, according to a first difference value between the first pixel point and the second pixel point in the characteristic value, a matching degree value between the first pixel point and the second pixel point comprises: Determine a first variance of the eigenvalues of all pixels within a preset range of the first pixel, and determine a second variance of the eigenvalues of all pixels within a preset range of the second pixel, so as to determine a second difference value between the first variance and the second variance; A first inverse of the first difference value is determined, and a second inverse of the second difference value is determined, so as to multiply the first inverse and the second inverse to obtain the matching degree value.
3. The automatic wheel set loading and unloading method based on machine vision according to claim 2, It is characterized in that The first difference value between the first pixel point and the second pixel point in the characteristic value is determined by the following method: , where M 1 is the first difference between the first pixel and the second pixel in the eigenvalue, A 1 is the eigenvalue of the first pixel, A 2 is the characteristic value of the second pixel, and T is a preset value greater than 0.
4. The method for automatic wheel set loading and unloading based on machine vision according to claim 2, It is characterized in that The second difference between the first variance and the second variance is determined as follows: , where M 2 is the second difference between the first variance and the second variance, D 1 is the first variance, D 2 is the second variance, and T is a preset value greater than 0.
5. The method for automatic wheel set loading and unloading based on machine vision according to claim 1, It is characterized in that The characteristic value of the first pixel point is determined based on the grayscale mean of all pixels within a preset range of the first pixel point in the first grayscale image; the characteristic value of the second pixel point is determined based on the grayscale mean of all pixels within a preset range of the second pixel point in the second grayscale image.
6. The automatic wheel set loading and unloading method based on machine vision according to claim 1, It is characterized in that Loading and unloading the target wheelset according to the first target image and the second target image includes: Performing edge detection on the first target image to obtain a first edge image; Performing edge detection on the second target image to obtain a second edge image; A first position where the center of the wheel disc of the target wheelset is located is determined according to the first edge image and the second edge image, and the target wheelset is loaded and unloaded according to the first position.
7. The automatic wheel set loading and unloading method based on machine vision according to claim 1, It is characterized in that The wheelset is provided with an inertial measurement unit, and the method further comprises: Receiving inertial measurement data collected by the inertial measurement unit, and obtaining an initial position of the wheelset moving in the wheelset parking area; Determine, according to the initial position and the inertial measurement data, the position information of the wheelset in the wheelset parking area at a current moment; According to different position information of different wheelsets in the parking area at the current moment, a wheelset position distribution map corresponding to the parking area at the current moment is generated.
8. The method for automatic wheel set loading and unloading based on machine vision according to claim 1, It is characterized in that The method further comprises: Acquire first point cloud data of a target area where a first wheelset to be loaded is located, and acquire a third target image of the target area; Matching the first point cloud data and the third target image to determine a second position of the first wheel pair to be loaded in space; A first control instruction is sent to the wheelset moving mechanism so that the wheelset moving mechanism moves the first wheelset to be loaded from the second position to the target position, where the first target vehicle for transporting the first wheelset to be loaded is parked.
9. The method for automatic wheel set loading and unloading based on machine vision according to claim 1, It is characterized in that The method further comprises: In the case of unloading the wheelset loaded in the second target vehicle, sending a third control instruction to the wheelset moving mechanism to move the wheelset moving mechanism to above the loading area of the second target vehicle, wherein the second target vehicle is a vehicle loaded with the wheelset to be unloaded; Acquire a fourth target image of the loading area, and acquire second point cloud data of the loading area; Determine, from the loading area, according to the second point cloud data and the fourth target image, a second wheelset having a minimum vertical distance from the wheelset moving mechanism; A fourth control instruction is sent to the wheelset moving mechanism so that the wheelset moving mechanism removes the second wheelset from the second target vehicle.
10. A wheel set automatic loading and unloading system based on machine vision, It is characterized in that include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the method for automatic loading and unloading of wheelsets based on machine vision according to any one of claims 1 to 9 is implemented.
