Guide wheel defect detection method, device, server and system

Abstract

The invention provides a guide wheel defect detection method, device, server and system. The method comprises the following steps: determining a to-be-detected image corresponding to any guide wheel of a vehicle; the to-be-detected image is an image obtained after an area where a non-hot-melt rubber material is located is recognized from a guide wheel image; and determining defect information of the guide wheel according to the to-be-detected image, and identifying an area where a non-hot-melt rubber material is located in the guide wheel image, so that the to-be-detected image of a normal guide wheel and the to-be-detected image of a hot-melt guide wheel have significant difference, thereby accurately determining whether the guide wheel has defects or not.

Description

Technical Field

[0001] The present invention relates to the technical field of tire detection, and in particular to a guide wheel defect detection method, device, server and system. Background Art

[0002] Trams play an important role in the suburbs of large cities, helping to alleviate traffic congestion and providing a fast, safe and convenient way to travel.

[0003] Trams use running wheels and guide wheels located below and inside the running wheels to travel and steer on the track beams. The guide wheels are a key component of the vehicle's steering system, working together with other structural components to control the vehicle's steering. The outer surface of the guide wheels is made of rubber. During the steering process, the guide wheels directly contact the inner rail beams, subjecting them to high friction. This can cause the rubber material to melt, damaging the appearance of the guide wheels, ultimately affecting driving safety and causing unusual noises in the vehicle, which can affect the passenger experience. Therefore, it is necessary to inspect the appearance of the guide wheels to determine if they have any defects. Summary of the Invention

[0004] The present invention provides a guide wheel defect detection method, device, server and system for detecting the appearance of a guide wheel to determine whether the guide wheel has defects.

[0005] In a first aspect, the present invention provides a guide wheel defect detection method, the method comprising:

[0006] Determine an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image;

[0007] Defect information of the guide wheel is determined based on the image to be detected.

[0008] Optionally, determining defect information of the guide wheel according to the image to be detected includes:

[0009] Feature information corresponding to at least one position in the image to be detected is extracted by a feature extractor, and defect information of the guide wheel is determined based on the extracted feature information.

[0010] Optionally, determining defect information of the guide wheel according to the extracted feature information includes:

[0011] For each position of feature information corresponding to the position, determine the nearest neighbor feature information corresponding to the feature information stored in the memory feature library;

[0012] Defect information of the guide wheel is determined based on the feature information corresponding to each position and the nearest neighbor feature information.

[0013] Optionally, determining the defect information of the guide wheel according to the feature information corresponding to each position and the nearest neighbor feature information includes:

[0014] For each position, determining difference information between the feature information corresponding to the position and the nearest feature information;

[0015] According to the difference information corresponding to each position, it is determined that the guide wheel has a defect.

[0016] Optionally, determining whether the guide wheel has a defect according to the difference information corresponding to each position includes:

[0017] For any position, determine the anomaly score of the position based on the difference information corresponding to the position;

[0018] When the abnormality score of any position is greater than a first threshold, it is determined that the guide wheel has a defect.

[0019] Optionally, the method further includes:

[0020] After determining that the guide wheel has a defect, the position where the abnormal value score is greater than the first threshold is determined as the defect position.

[0021] Optionally, the memory feature library is determined by:

[0022] Acquire multiple images of guide wheels without defects, and obtain corresponding normal samples by identifying the area where the rubber material is located in the images of the guide wheels without defects;

[0023] For each normal sample, extracting feature information corresponding to different positions in the normal sample by the feature extractor;

[0024] The memory feature library is determined according to each piece of feature information extracted.

[0025] Optionally, the feature extractor includes a first feature extractor and a second feature extractor; extracting feature information corresponding to different positions in the normal sample by the feature extractor includes:

[0026] determining first feature information corresponding to each position in the normal sample by the first feature extractor, and determining second feature information corresponding to each position in the normal sample by the second feature extractor; the number of convolution layers of the first feature extractor is different from the number of convolution layers of the second feature extractor;

[0027] The first feature information and the second feature information corresponding to the same position are fused to obtain feature information corresponding to the position.

[0028] Optionally, determining the memory feature library according to each extracted feature information includes:

[0029] Cluster the extracted feature information;

[0030] The memory feature library is determined according to the clustered plurality of feature information.

[0031] Optionally, determining the memory feature library based on the clustered plurality of feature information includes:

[0032] For each category obtained after clustering, the feature information corresponding to the category is subjected to dimensionality reduction processing;

[0033] Each feature information after the dimensionality reduction process is determined as the feature information in the memory feature library.

[0034] Optionally, determining defect information of the guide wheel according to the image to be detected includes:

[0035] Defect information of the guide wheel is determined according to the connected domain in the image to be detected.

[0036] Optionally, determining defect information of the guide wheel according to a connected domain in the image to be detected includes:

[0037] Determining the number of connected domains in the image to be detected;

[0038] When the number of the connected domains is greater than a second threshold, it is determined that the guide wheel has a defect.

[0039] Optionally, the method further includes:

[0040] After determining that the guide wheel has a defect, predicting the circular area where the guide wheel is located based on each connected domain existing in the image to be detected;

[0041] The areas in the circular area except the connected domains are determined as defect locations.

[0042] Optionally, determining an image to be detected corresponding to any guide wheel of the vehicle includes:

[0043] Identifying the area where the non-hot-melt rubber material is located in the guide wheel image by using a recognition model to obtain an image to be detected;

[0044] Among them, the recognition model is a model obtained by training based on the target data set; the target data set is a data set composed of defect-free guide wheel images, which is used to train the recognition model to identify the area where the rubber material of the guide wheel image in the target training set is located.

[0045] Optionally, the recognition model is a YOLO model; the detection head in the YOLO model includes a detection head for segmentation detection and does not include a detection head for target detection.

[0046] Optionally, after determining the defect information of the guide wheel, the method further includes:

[0047] Entering the defect information into a database;

[0048] And / or, the defect information and the corresponding guide wheel image are sent to a display terminal, so that a reviewer can review the defect information based on the guide wheel image.

[0049] Optionally, the method is applied to a server; the vehicle runs on two track beams, and an image acquisition device is provided at a preset position between the two track beams, and the image acquisition device is used to capture images of the vehicle passing by; the server is used to receive the image sent by the image acquisition device and generate an image of the bottom of the vehicle; the guide wheel image is an image of the guide wheel segmented from the image of the bottom of the vehicle.

[0050] Optionally, an approaching vehicle detection device and a leaving vehicle detection device are provided within a preset range of the image acquisition device; the approaching vehicle detection device is used to control the image acquisition device to start capturing images when it detects the arrival of a vehicle; the leaving vehicle detection device is used to control the image acquisition device to stop capturing images when it detects the departure of a vehicle.

[0051] Optionally, the image acquisition device is a line array camera, and the images captured by the line array camera are multiple local images of the bottom of the vehicle when the vehicle is running; the bottom image of the vehicle is an image obtained by splicing the multiple local images.

[0052] In a second aspect, the present invention provides a guide wheel defect detection device, the device comprising:

[0053] The first determination module is used to determine the image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image;

[0054] The second determining module is used to determine the defect information of the guide wheel according to the image to be detected.

[0055] In a third aspect, the present invention provides a server comprising: at least one processor and a memory;

[0056] The memory stores computer-executable instructions;

[0057] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method as described in any one of the first aspects.

[0058] In a fourth aspect, the present invention provides a detection system comprising:

[0059] An image acquisition device is used to capture multiple partial images of the bottom of the vehicle while the vehicle is running, so as to obtain a vehicle bottom image and a guide wheel image; the vehicle bottom image is an image obtained by stitching together the multiple partial images; the guide wheel image is an image of the guide wheel segmented from the vehicle bottom image;

[0060] An approaching vehicle detection device, configured to control the image acquisition device to start capturing images upon detecting an approaching vehicle;

[0061] A vehicle departure detection device is used to control the image acquisition device to stop capturing images when it detects that the vehicle has left.

[0062] The server is configured to determine an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying an area where a non-hot-melt rubber material is located from the guide wheel image;

[0063] Defect information of the guide wheel is determined based on the image to be detected.

[0064] In a fifth aspect, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in any one of the first aspects is implemented.

[0065] In a sixth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0066] The present invention provides a guide wheel defect detection method, device, server and system, which include: determining an image to be detected corresponding to any guide wheel of a vehicle; the image to be detected is an image obtained after identifying the area where non-hot-melt rubber material is located from the guide wheel image; determining the defect information of the guide wheel based on the image to be detected, and by identifying the area where non-hot-melt rubber material is located in the guide wheel image, there will be significant differences in the images to be detected for a normal guide wheel and a guide wheel that has undergone hot melting, thereby accurately determining whether the guide wheel has a defect. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0068] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention;

[0069] Figure 2 A schematic flow chart of a guide wheel defect detection method is provided for an embodiment of the present invention;

[0070] Figure 3 A schematic diagram of an image to be inspected corresponding to an image of a guide wheel without defects provided by an embodiment of the present invention;

[0071] Figure 4 A schematic diagram of an image to be inspected corresponding to an image of a defective guide wheel provided in an embodiment of the present invention;

[0072] Figure 5 A schematic diagram of an overall system provided by an embodiment of the present invention;

[0073] Figure 6 A schematic diagram of a partial image captured by an image acquisition device provided by an embodiment of the present invention;

[0074] Figure 7 A schematic diagram of a vehicle bottom image provided by an embodiment of the present invention;

[0075] Figure 8 A schematic diagram of a first-level positioning image provided by an embodiment of the present invention;

[0076] Figure 9 An overall structural recognition diagram of a recognition model provided by an embodiment of the present invention;

[0077] Figure 10 A schematic flow chart of another guide wheel defect detection method provided by an embodiment of the present invention;

[0078] Figure 11 A defect detection result of a defect-free guide wheel image provided by an embodiment of the present invention;

[0079] Figure 12 A defect detection result of a defective guide wheel image provided by an embodiment of the present invention;

[0080] Figure 13 A schematic diagram of obtaining a memory feature library corresponding to a single normal sample provided by an embodiment of the present invention;

[0081] Figure 14 A schematic structural diagram of a guide wheel defect detection device 140 provided in an embodiment of the present invention;

[0082] Figure 15 A schematic diagram of the hardware structure of a server provided in an embodiment of the present invention.

[0083] The above drawings illustrate specific embodiments of the present invention, which will be described in more detail below. These drawings and the accompanying description are not intended to limit the scope of the present invention in any way, but rather to illustrate the concept of the present invention to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present invention.

[0085] In this document, it should be understood that the terms involved are only used to facilitate understanding and do not represent any limitation on the meaning. In addition, the number of any element in the drawings is for illustration and not limitation, and any naming is only for distinction and does not have any limiting meaning.

[0086] The data involved in the present invention may be data authorized by the user or fully authorized by all parties, and the collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations.

[0087] Steering wheels are a crucial component in vehicles, used to control steering. These wheels are made of rubber. During vehicle operation, friction from the wheel's contact with the track beam can cause the rubber to melt, compromising safe driving. Therefore, it's desirable to detect whether steering wheels in vehicles are melting, thereby improving vehicle safety.

[0088] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention, such as Figure 1 As shown, the method can be applied in a server, or it can also be a processing device with data computing capabilities. The server can process the image of the guide wheel when the vehicle is running, identify the area of non-hot-melt rubber material in the guide wheel image, and determine the image to be detected, so that the images to be detected corresponding to the guide wheel with hot-melt phenomenon and the guide wheel without hot-melt phenomenon are different, thereby determining whether the guide wheel has defects based on the image to be detected.

[0089] Figure 2 A flow chart of a guide wheel defect detection method is provided for an embodiment of the present invention, as shown in FIG. Figure 2 As shown, the method includes:

[0090] Step S201, determining an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image.

[0091] While a vehicle is operating on a track beam, an image acquisition device can be used to capture images of the vehicle in operation. To detect defects in the guide wheels, the captured images include only the guide wheels. Optionally, if a vehicle includes multiple guide wheels, the captured images can include a complete image of all guide wheels to detect defects in each guide wheel. Because the guide wheels need to be inspected, images can be captured that only include each guide wheel, thereby generating guide wheel images.

[0092] After determining the guide wheel image, since it is necessary to detect whether the guide wheel is hot-melted, the area where the non-hot-melt rubber material is located can be identified from the guide wheel image to obtain the image to be detected.

[0093] Figure 3 A schematic diagram of an image to be inspected corresponding to an image of a guide wheel without defects provided by an embodiment of the present invention. Figure 4 A schematic diagram of an image to be detected corresponding to an image of a defective guide wheel provided in an embodiment of the present invention is shown as follows: Figure 3 As shown in FIG, there is no rubber hot melt defect in the guide wheel, and the position corresponding to the guide wheel in the image to be detected is a complete circle; Figure 4 As shown, there is a rubber hot melt defect in the guide wheel, and the position corresponding to the guide wheel in the obtained image to be detected is an incomplete circle.

[0094] That is to say, when there is a hot melt defect in the guide wheel, only the area where the non-hot melt rubber material is located can be identified, but the area where the hot melt rubber material is located cannot be identified, so that the images to be detected corresponding to the defect-free guide wheel image and the defective guide wheel image are different, so that it is possible to identify whether the guide wheel has a hot melt defect based on the image to be detected.

[0095] Step S202: Determine defect information of the guide wheel according to the image to be detected.

[0096] Since the images to be detected corresponding to the two types of images are significantly different, the defect information of the guide wheel can be determined based on the images to be detected.

[0097] The present invention provides a guide wheel defect detection method, which includes: determining an image to be detected corresponding to any guide wheel of a vehicle; the image to be detected is an image obtained after identifying the area where non-hot-melt rubber material is located from the guide wheel image; determining the defect information of the guide wheel based on the image to be detected, and by identifying the area where non-hot-melt rubber material is located in the guide wheel image, there will be significant differences in the images to be detected for a normal guide wheel and a guide wheel that has undergone hot melting, thereby accurately determining whether the guide wheel has a defect.

[0098] The following describes the entire implementation process, which includes image preprocessing, guide wheel positioning, guide wheel defect detection, and entering the detection results into the database. First, the process of generating the guide wheel image is described, namely the image preprocessing, guide wheel positioning, and entering the detection results into the database.

[0099] Optionally, the method is applied to a server; the vehicle runs on two track beams, and an image acquisition device is provided at a preset position between the two track beams, and the image acquisition device is used to capture images of the vehicle passing by; the server is used to receive the image sent by the image acquisition device and generate an image of the bottom of the vehicle; the guide wheel image is an image of the guide wheel segmented from the image of the bottom of the vehicle.

[0100] Optionally, the image acquisition device is a line array camera, and the images captured by the line array camera are multiple local images of the bottom of the vehicle when the vehicle is running; the bottom image of the vehicle is an image obtained by splicing the multiple local images.

[0101] Figure 5 A schematic diagram of an overall system provided by an embodiment of the present invention is shown as follows: Figure 5 As shown, the method of the present application can be applied to a server, which can obtain images captured by an image acquisition device and process the acquired images to output defect information corresponding to the vehicle. The vehicle runs on track beams, and an image acquisition device can be set in the middle position between the two track beams to capture images of the vehicle during operation.

[0102] The image acquisition device can be a line scan camera, which has a high resolution and can capture high-resolution images, thereby improving the accuracy of subsequent defect detection. The image captured by the image acquisition device is a partial image. After acquiring the image, the image acquisition device can send the acquired image to the server. The server can stitch together multiple partial images to obtain a stitched image, that is, an image of the vehicle bottom.

[0103] Optionally, after acquiring multiple partial images, the server may perform preprocessing operations on the acquired images. Preprocessing operations may include blank image removal, filtering, image enhancement, image stitching, etc. Optionally, since the triggering time of the image acquisition device is usually earlier than the time when the vehicle arrives at the installation position of the image acquisition device, the image acquisition device will capture a blank image, and the blank image in the acquired image may be removed. The acquired partial image may contain noise, and the area of interest cannot be presented in a single image. Therefore, the acquired image may be filtered. Since the acquired image is an image of the bottom of the vehicle, the image is dark overall, and the acquired image may be enhanced to adjust the overall brightness of the image. Since the acquired image is a partial image of the bottom of the vehicle, the multiple acquired images may be stitched together to obtain an image of the bottom of the vehicle.

[0104] Figure 6 A schematic diagram of a partial image captured by an image acquisition device according to an embodiment of the present invention is shown in FIG. Figure 6 As shown, the captured image is a partial image of the bottom of the vehicle. Figure 7 A schematic diagram of a vehicle bottom image provided by an embodiment of the present invention is shown as follows: Figure 7 As shown in FIG, after preprocessing, a complete vehicle bottom image is obtained.

[0105] By installing an image acquisition device at a preset position on the track beam, it is possible to capture the bottom image of the vehicle while the vehicle is running and detect in real time whether there are any abnormalities in the guide wheels. This allows the guide wheels to be detected for defects without having to be removed from the vehicle.

[0106] Optionally, an approaching vehicle detection device and a leaving vehicle detection device are provided within a preset range of the image acquisition device; the approaching vehicle detection device is used to control the image acquisition device to start capturing images when it detects the arrival of a vehicle; the leaving vehicle detection device is used to control the image acquisition device to stop capturing images when it detects the departure of a vehicle.

[0107] Optionally, the image acquisition device may be disposed in a caisson, thereby protecting the image acquisition device.

[0108] In order to ensure that the image acquisition device can accurately capture the image of the vehicle when it is running, an oncoming vehicle detection device can be set within the preset range of the image acquisition device ( Figure 5 When the vehicle detection device detects the arrival of a vehicle, it can control the car door to open and control the image acquisition device to start taking images.

[0109] Optionally, a vehicle-leaving detection device ( Figure 5When the train leaving system sleep identification device detects that the vehicle has left, the image acquisition device can be controlled to stop taking images, and the caisson door can also be controlled to close.

[0110] Optionally, since the image acquisition device captures images of the bottom of the vehicle and the shooting environment is dark, a fill light device can be provided in the caisson to implement fill light when the image acquisition device is capturing images.

[0111] Figure 5 The image acquisition device installed on the central gantry can capture images of the vehicle's appearance to detect defects. The figure also involves a license plate recognition camera, speed measurement device, and vehicle receiving unit, which will not be detailed here.

[0112] After determining the vehicle bottom image, the image of the guide wheel region can be segmented. Alternatively, the guide wheel region can be located through step-by-step segmentation and threshold segmentation, thereby obtaining a guide wheel image. Alternatively, a primary positioning result can be obtained first, followed by a secondary positioning result, with the secondary positioning result being the guide wheel image. Figure 8 A schematic diagram of an image after primary positioning provided by an embodiment of the present invention, including four guide wheels and images of objects around the guide wheels. Figure 3 The image of the guide wheel without defects on the left is the secondary positioning result. Figure 4 The image of the defective guide wheel on the left is the secondary positioning result.

[0113] By providing an oncoming vehicle detection device and a leaving vehicle detection device, it is possible to control whether the image acquisition device captures images, so that the image acquisition device captures images when a vehicle passes by.

[0114] Optionally, after determining the defect information of the guide wheel, the method further includes:

[0115] Entering the defect information into a database;

[0116] And / or, the defect information and the corresponding guide wheel image are sent to a display terminal, so that a reviewer can review the defect information based on the guide wheel image.

[0117] After the defect information of the guide wheel is determined by the server, the defect information may specifically include whether the vehicle has a guide wheel defect and / or the defect location.

[0118] After the defect information is determined, the defect information can be entered into the database for data storage. Figure 5As shown, the defect information can also be sent to a display terminal for display. The display terminal can be a web terminal, such as a PC (Personal Computer) terminal or a mobile APP (Application) terminal. For example, the reviewer of the staff can obtain the defect information based on the display terminal, so that the defect information can be reviewed to improve the accuracy of the determined defect information, so that the defective guide wheel can be replaced in time.

[0119] By displaying and storing the defect information in a database, it is possible to facilitate the review party such as the staff to continue to process the defect information, so as to improve the accuracy of the determined defect information.

[0120] The above describes the process of generating the guide wheel image and the processing after determining the defect information. The following describes the process of determining the image to be detected from the guide wheel image, that is, the guide wheel defect detection process.

[0121] Optionally, determining an image to be detected corresponding to any guide wheel of the vehicle includes:

[0122] The area where the non-hot-melt rubber material is located in the guide wheel image is identified by a recognition model to obtain an image to be detected; wherein, the recognition model is a model trained based on a target data set; the target data set is a data set composed of defect-free guide wheel images, which is used to train the recognition model to identify the area where the rubber material of the guide wheel image in the target training set is located.

[0123] Determining the image to be detected corresponding to the guide wheel image is essentially to identify and extract the area where the non-hot-melt rubber material is located in the guide wheel image through a recognition model. Optionally, the recognition model is a rubber material extraction model constructed by a supervised deep learning semantic segmentation model, which is used to extract the rubber material of the guide wheel. Optionally, the recognition model can be a model trained based on a data set composed of defect-free guide wheel images. By using the defect-free guide wheel image training model, when the recognition model determines the image to be detected from the guide wheel image, it can obtain the area where the non-hot-melt rubber material is located from the guide wheel image, instead of obtaining the area where the hot-melt rubber material is located, thereby making it possible to distinguish between the image to be detected corresponding to the guide wheel image with hot-melt defects and the image to be detected corresponding to the guide wheel image without hot-melt defects.

[0124] By extracting the area where the non-hot-melt rubber material is located in the guide wheel image, the influence of the noisy background on the area of interest can be removed. The area of interest here refers to the area where the rubber material of the guide wheel is located.

[0125] After the target data set is used to train the recognition model, the trained recognition model can be saved and used to extract the rubber material, that is, to determine the image to be detected of the guide wheel. Figure 3 and Figure 4 As shown in the figure, the recognition model can effectively extract the peripheral rubber structure of the guide wheel in different scenarios.

[0126] Optionally, the recognition model is a YOLO model; the detection head in the YOLO model includes a detection head for segmentation detection and does not include a detection head for target detection.

[0127] The recognition model can be implemented using a YOLO model, and optionally, a YOLOv8 model. In the scenario of this application, the recognition model is used to extract the area where the rubber material is located, without the need for target detection. Therefore, the detection head in the YOLO model can only be equipped with a segmentation detection head to segment the area where the rubber material is located from the guide wheel image, without setting a target detection head, so as not to identify and mark the target, thereby reducing the computational complexity of the YOLO model and increasing the speed at which the recognition model outputs the image to be detected.

[0128] Figure 9 The overall structure recognition diagram of a recognition model provided by an embodiment of the present invention is as follows: Figure 9 As shown in the figure, the model is a YOLO model, and the detection head in the model only contains seg, which is the detection head for segmentation detection.

[0129] Furthermore, the loss function used in the recognition model in this application can adopt the FocalLoss loss function. This loss function is computationally lightweight, while existing composite loss functions require a large amount of computation, which can reduce the real-time performance of the recognition model output. Furthermore, in the image to be detected, the rubber material area accounts for a relatively small proportion of the entire image, indicating a sample imbalance. The FocalLoss loss function can address this sample imbalance and improve the accuracy of the recognition model output.

[0130] By using a dataset consisting of defect-free guide wheel images to train the YOLO model to obtain a recognition model, the recognition model can identify and extract non-hot-melt rubber material areas. In addition, by improving the YOLO model, the recognition model can quickly and accurately obtain output results.

[0131] The above describes the process of determining the image to be inspected. The following describes the process of determining whether there is a defect based on the image to be inspected. Here, when determining whether the guide wheel has a defect, there are only two methods to achieve it. Figure 10 A flow chart of another guide wheel defect detection method provided by an embodiment of the present invention is shown as follows: Figure 10 As shown, the method includes: step S301, extracting the area where the tire rubber material is located in the image to be detected (if the guide wheel image is a defect-free image, the area extracted is the area where the rubber material is located in the image; if the guide wheel image is a defective image, the area extracted is the area where the non-hot-melt rubber material is located in the image). Defect detection can be performed based on two methods. One method is: step S302, building a memory feature library based on normal samples, step S303, performing intelligent diagnosis based on the memory feature library, that is, performing defect detection; one method is: step S302, extracting connected domains, step S303, performing intelligent diagnosis based on the number of connected domains, that is, performing defect detection. Among them, in the first method mentioned above, the process of building the memory feature library can be completed in advance and executed once, without having to be executed once each time the guide wheel image is subjected to defect detection. The first defect detection method is described below.

[0132] Optionally, determining defect information of the guide wheel according to the image to be detected includes:

[0133] Feature information corresponding to at least one position in the image to be detected is extracted by a feature extractor, and defect information of the guide wheel is determined based on the extracted feature information.

[0134] After determining the image to be inspected, feature information of at least one location in the image to be inspected can be extracted, and based on the feature information, whether the guide wheel has a defect can be determined. Specifically, if the guide wheel is heat-melted, the feature information of the heat-melted portion differs from the feature information of the non-heat-melted portion. Based on this characteristic, whether the guide wheel has a defect can be determined based on the feature information of at least one location in the image to be inspected.

[0135] Alternatively, if only the guide wheel needs to be determined to be defective, it is not necessary to extract feature information for every location in the image to be inspected. Feature information can be extracted from only some locations, where heat-melting may occur. If the feature information corresponding to at least one location meets certain conditions, the guide wheel can be determined to be defective.

[0136] By determining whether the guide wheel has defects based on characteristic information corresponding to at least one position, and based on the fact that the characteristic information of the hot-melt portion is different from the characteristic information of the portion where no hot-melt occurs, it is possible to accurately determine whether the guide wheel has defects, thereby improving the accuracy of determining whether the guide wheel has defects.

[0137] Optionally, determining defect information of the guide wheel according to the extracted feature information includes:

[0138] For each position of feature information corresponding to the position, determine the nearest neighbor feature information corresponding to the feature information stored in the memory feature library;

[0139] Defect information of the guide wheel is determined based on the feature information corresponding to each position and the nearest neighbor feature information.

[0140] Based on the extracted feature information, the nearest neighbor feature information corresponding to the feature information can be retrieved from the memory feature library. Optionally, the memory feature library stores feature information derived from a normal sample. A normal sample, in this context, refers to an image of a guide wheel without defects, where the rubber material is located. In other words, the feature information stored in the memory feature library represents a normal sample. Therefore, the nearest neighbor feature information can be determined based on the extracted feature information, and the presence of a defect at that location can be determined based on the feature information and the nearest neighbor feature information.

[0141] Optionally, the distance between each feature information in the memory feature library and the extracted feature information may be calculated, and the feature information with the shortest distance may be determined as the nearest neighbor feature information of the extracted feature information.

[0142] Optionally, when determining the nearest neighbor feature information, a kd tree or octree search method may be used to obtain the corresponding nearest neighbor feature. By searching the nearest neighbor feature information using the above method, the nearest neighbor feature can be quickly found.

[0143] Optionally, determining the defect information of the guide wheel according to the feature information corresponding to each position and the nearest neighbor feature information includes:

[0144] For each position, determining difference information between the feature information corresponding to the position and the nearest feature information;

[0145] According to the difference information corresponding to each position, it is determined that the guide wheel has a defect.

[0146] After determining the nearest neighbor feature information corresponding to the feature information at each position, the feature information at each position may be processed to determine whether the guide wheel has a defect.

[0147] For a position, the difference information between the feature information corresponding to the position and the corresponding nearest neighbor feature information can be calculated. The difference information can reflect the degree of deviation between the two feature information, so as to determine the gap between the position and similar positions in normal samples.

[0148] Optionally, the difference information may be a difference between the feature information and a corresponding position in the corresponding nearest neighbor feature information, so that based on the difference it may be determined whether there is a defect at the position in the guide wheel.

[0149] Optionally, determining whether the guide wheel has a defect according to the difference information corresponding to each position includes:

[0150] For any position, determine the anomaly score of the position based on the difference information corresponding to the position;

[0151] When the abnormality score of any position is greater than a first threshold, it is determined that the guide wheel has a defect.

[0152] After determining the difference information for a particular location, an outlier score for that location can be determined. Optionally, the difference information can be the difference between the feature information corresponding to that location and the corresponding location of the nearest neighbor feature information. This difference is a multidimensional vector. This difference can then be averaged, and the averaged result is used as the outlier score. This outlier score measures the degree of difference between the feature information at that location and the corresponding nearest neighbor feature information.

[0153] Optionally, a first threshold may be set. When the outlier score is greater than the first threshold, it indicates that a defect exists at the location.

[0154] Optionally, when the abnormality score of at least one position is greater than a first threshold, it can be determined that the guide wheel has a defect. The specific value of the first threshold is not limited here and can be set according to actual conditions.

[0155] Figure 11 The defect detection result of a defect-free guide wheel image provided by an embodiment of the present invention is: Figure 12 This is a defect detection result of a defective guide wheel image provided by an embodiment of the present invention. As can be seen from the figure, the above method can well detect hot melt defects.

[0156] By determining the nearest neighbor features and outlier value scores as described above, it is possible to accurately determine whether there is a defect at each position in the guide wheel. When there are tiny cracks in the guide wheel, the above method can also be used to effectively detect them with high detection accuracy.

[0157] Optionally, the method further includes:

[0158] After determining that the guide wheel has a defect, the position where the abnormal value score is greater than the first threshold is determined as the defect position.

[0159] When it is determined that the guide wheel has a defect, the defect position can also be determined at the same time, and the position where the abnormal value score is greater than the first threshold can be determined as the defect position.

[0160] In practice, feature information of each position in the image to be inspected can be determined, thereby detecting whether each position is a defect position.

[0161] The above method can accurately locate the defect position in the guide wheel, making it easier for auditors such as staff to review whether each defect position actually has a defect.

[0162] The above method requires the use of a memory feature library. Therefore, before using the above method for defect detection, the memory feature library may be determined.

[0163] Optionally, the memory feature library is determined by:

[0164] Acquire multiple images of guide wheels without defects, and obtain corresponding normal samples by identifying the area where the rubber material is located in the images of the guide wheels without defects;

[0165] For each normal sample, extracting feature information corresponding to different positions in the normal sample by the feature extractor;

[0166] The memory feature library is determined according to each piece of feature information extracted.

[0167] When determining the memory library, multiple defect-free guide wheel images can be obtained first. For each defect-free guide wheel image, the area where the rubber material is located can be identified, thereby obtaining the corresponding normal sample, that is, obtaining Figure 3 Normal samples similar to the image to be detected are shown.

[0168] When multiple images of defect-free guide wheels are available, multiple normal samples can be obtained. For each normal sample, the feature extractor can extract feature information corresponding to different locations in the normal sample. Because the extracted feature information is extracted from the normal sample, the memory feature library determined based on the extracted feature information can reflect the feature information at different locations in the normal sample.

[0169] The memory feature library determined by the above method can obtain feature information corresponding to different positions in the normal sample, so that whether there is a defect at a certain position in the image to be inspected can be detected based on the memory feature library.

[0170] Optionally, the feature extractor includes a first feature extractor and a second feature extractor; extracting feature information corresponding to different positions in the normal sample by the feature extractor includes:

[0171] determining first feature information corresponding to each position in the normal sample by the first feature extractor, and determining second feature information corresponding to each position in the normal sample by the second feature extractor; the number of convolution layers of the first feature extractor is different from the number of convolution layers of the second feature extractor;

[0172] The first feature information and the second feature information corresponding to the same position are fused to obtain feature information corresponding to the position.

[0173] Figure 13 This is a schematic diagram of obtaining a memory feature library corresponding to a single normal sample provided by an embodiment of the present invention. Figure 13 As shown, in order to improve the accuracy of the extracted feature information, the first feature extractor and the second feature extractor can be used to extract the feature information of each position in the normal sample respectively to obtain the first feature information and the second feature information, and the first feature information and the second feature information corresponding to the same position are fused to obtain the feature information corresponding to the position, thereby obtaining the feature information corresponding to each position in the normal sample image.

[0174] Optionally, the first feature extractor may be ResNet50 Stage2, and the second feature extractor may be ResNet50 Stage3. A feature map corresponding to a normal sample may be obtained based on the first feature extractor, and a feature map corresponding to a normal sample may be obtained based on the second feature extractor. After obtaining the two feature maps, for each feature map, the following operation may be performed on each position in the feature map: for the current position, the features within a neighborhood with a radius of ρ corresponding to the current position are aggregated to obtain the local features of the current position.

[0175] Since the sizes of the feature information extracted by the two feature extractors are different, the results obtained by the second feature extractor can be upsampled so that the feature information obtained by the second feature extractor has the same resolution as the feature information obtained by the first feature extractor, so that the feature information of the corresponding positions obtained by the two feature extractors can be fused to obtain the fused feature information of each position, thereby obtaining the feature information corresponding to a single normal sample.

[0176] Optionally, you can set φ j+1 (x i ) after bilinear interpolation and φ j (x i ) to maintain the same resolution, then splice the two together, and then use adaptive mean pooling to generate the memory feature information of a single image.

[0177] For each normal sample, the above method can be used to determine the corresponding feature information. The specific method is as follows:

[0178]

[0179] Optionally, the first feature extractor and the second feature extractor can be pre-trained based on a public data set, or can be trained using normal samples of the present application, so that the first feature extractor and the second feature extractor can extract feature information well.

[0180] It should be noted that, when determining the feature information corresponding to at least one position in the image to be detected, the feature information extracted by the two feature extractors mentioned above is also used.

[0181] By setting two feature extractors with different numbers of convolutional layers, more accurate feature information can be extracted compared to setting one feature extractor.

[0182] Optionally, determining the memory feature library based on each extracted feature information includes:

[0183] Cluster the extracted feature information;

[0184] The memory feature library is determined according to the clustered plurality of feature information.

[0185] The memory feature library M generated by the entire training set contains N*dim data. As the number of training set samples increases, its value grows linearly, which seriously consumes memory and computing time.

[0186] After determining the feature information corresponding to each normal sample, since there are multiple normal samples, the feature information may contain the same or similar feature information. In order to reduce the amount of feature information stored in the memory feature library, part of the feature information in the memory feature library can be cropped to reduce memory consumption and improve the processing speed when determining the nearest neighbor features.

[0187] Optionally, the feature information extracted from each normal sample can be clustered to reduce the amount of feature information in the memory feature library. Optionally, a K-means clustering method can be used for clustering, and then a feature information from a category is selected to be retained, while the other feature information in the category is deleted, thereby achieving the feature information in the memory feature library to be trimmed to find l features that best describe different locations in the normal sample.

[0188] Through the above operation, the amount of feature information in the memory feature library can be reduced, the memory consumption can be reduced, and the nearest neighbor feature information can be quickly determined, so that the entire defect detection time is greatly shortened.

[0189] Optionally, determining the memory feature library based on the clustered plurality of feature information includes:

[0190] For each category obtained after clustering, the feature information corresponding to the category is subjected to dimensionality reduction processing;

[0191] Each feature information after the dimensionality reduction process is determined as the feature information in the memory feature library.

[0192] A feature information can be obtained for each category after clustering, and dimensionality reduction processing can be performed based on the feature information. Optionally, a single-layer perceptron can be used to perform dimensionality reduction processing on each feature information to further reduce memory consumption.

[0193] Each feature information after dimensionality reduction is stored in the memory feature library to obtain the final memory feature library. Optionally, a kd tree or octree can be used to store the memory feature library to speed up the algorithm during inference.

[0194] Dimensionality reduction can further reduce memory consumption and speed up the algorithm.

[0195] The second method for determining defect information based on the image to be inspected is described below.

[0196] Optionally, determining defect information of the guide wheel according to the image to be detected includes:

[0197] Defect information of the guide wheel is determined according to the connected domain in the image to be detected.

[0198] exist Figure 3 and Figure 4 In the two images to be tested shown, it can be seen that when the rubber material is hot-melted, the rubber material of the hot-melt part cannot be extracted, so the image to be tested corresponding to the defective guide wheel image is not a complete circular area, and multiple connected domains can be formed. Therefore, it is possible to determine whether the guide wheel has defects based on the connected domains in the image to be tested.

[0199] Based on the differences in the connected domains in the image to be detected, it is possible to simply and conveniently determine whether the guide wheel has defects.

[0200] Optionally, determining defect information of the guide wheel according to a connected domain in the image to be detected includes:

[0201] Determining the number of connected domains in the image to be detected;

[0202] When the number of the connected domains is greater than a second threshold, it is determined that the guide wheel has a defect.

[0203] When determining whether the guide wheel exists based on the connected domain in the image to be detected, it can be determined based on the number of connected domains. When the guide wheel has not been melted, that is, there is no defect, the connected domain in the image to be detected is a complete circle, and the number of connected domains is one; when the guide wheel has been melted, that is, there is a defect, the connected domain in the image to be detected cannot form a complete circle, and the number of connected domains can be multiple. Figure 4 As shown in FIG, when there is a defect in the guide wheel, the number of connected domains in the image to be detected is 3.

[0204] Optionally, for the image to be detected, a connected domain acquisition algorithm may be used to find connected domains in the image to be detected, and the number of connected domains may be determined.

[0205] Optionally, a second threshold may be set. For example, the second threshold may be set to 1 or 2. When the number of connected domains is greater than the second threshold, it is determined that the guide wheel is defective. This application does not specifically limit the value of the second threshold.

[0206] By determining whether the guide wheel has defects based on the number of connected domains in the image to be inspected, the accuracy of determining whether the defects exist can be improved.

[0207] Optionally, the method further includes:

[0208] After determining that the guide wheel has a defect, predicting the circular area where the guide wheel is located based on each connected domain existing in the image to be detected;

[0209] The areas in the circular area except the connected domains are determined as defect locations.

[0210] After determining that a guide wheel defect exists, the defect location can also be determined. Optionally, for each detected connected domain, a circular region containing the guide wheel can be obtained. Specifically, each connected domain can be sequentially connected with arcs to form a circular region. After determining the circular region, the area outside each connected domain within the circular region can be identified as the defect location.

[0211] By determining the circular area where the guide wheel is located, the defect position can be accurately located, so that auditors such as staff can review the defect position and improve the user experience.

[0212] The connected domain method can detect areas where large-scale heat melting occurs, but cannot detect tiny crack defects. Depending on the specific scenario, you can choose to use method one or method two for defect detection.

[0213] The above-mentioned method for detecting guide wheel defects provided in the present application can detect the appearance of the tires of a running tram, can detect abnormal conditions of the tires during the operation of the vehicle, can improve the safety tolerance of the vehicle, and prevent abnormal vehicle noise caused by guide wheel abnormalities in advance.

[0214] The method for detecting guide wheel defects provided in this application can detect tire anomalies in real time while the vehicle is in operation, eliminating the need to remove the tire for testing. This saves labor costs and significantly improves detection efficiency. Furthermore, in this defect detection method, only normal samples are used to determine the recognition model and construct the memory feature library. Defective samples are only used for algorithm testing, eliminating the need to collect a large number of tire defect samples and reducing the reliance on defect sample data, thus resolving the difficulty of collecting defect samples.

[0215] The above-mentioned method for detecting guide wheel defects provided in the present application can detect defects of unknown forms, while most of the existing technologies are based on characterization and a specific fault as the basis for judgment. When the fault is undefined or abnormal, the fault cannot be detected. Therefore, compared with the existing technology, this method can well detect defects of unknown forms.

[0216] The above-mentioned method for detecting guide wheel defects provided in this application can share the same set of hardware equipment with the existing trackside online detection system, without the need for additional hardware cost expenditure.

[0217] Figure 14 This is a schematic structural diagram of a guide wheel defect detection device 140 provided in an embodiment of the present invention, the device comprising:

[0218] The first determining module 1401 is configured to determine an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image;

[0219] The second determining module 1402 is configured to determine defect information of the guide wheel according to the image to be detected.

[0220] Optionally, when determining the defect information of the guide wheel according to the image to be detected, the second determining module 1402 is specifically configured to:

[0221] Feature information corresponding to at least one position in the image to be detected is extracted by a feature extractor, and defect information of the guide wheel is determined based on the extracted feature information.

[0222] Optionally, when determining the defect information of the guide wheel according to the extracted feature information, the second determining module 1402 is specifically configured to:

[0223] For each position of feature information corresponding to the position, determine the nearest neighbor feature information corresponding to the feature information stored in the memory feature library;

[0224] Defect information of the guide wheel is determined based on the feature information corresponding to each position and the nearest neighbor feature information.

[0225] Optionally, when determining the defect information of the guide wheel according to the feature information corresponding to each position and the nearest neighbor feature information, the second determining module 1402 is specifically configured to:

[0226] For each position, determining difference information between the feature information corresponding to the position and the nearest feature information;

[0227] According to the difference information corresponding to each position, it is determined that the guide wheel has a defect.

[0228] Optionally, when determining that the guide wheel has a defect based on the difference information corresponding to each position, the second determining module 1402 is specifically configured to:

[0229] For any position, determine the anomaly score of the position based on the difference information corresponding to the position;

[0230] When the abnormality score of any position is greater than a first threshold, it is determined that the guide wheel has a defect.

[0231] Optionally, the device further includes: a first defect location determination module, configured to:

[0232] After determining that the guide wheel has a defect, the position where the abnormal value score is greater than the first threshold is determined as the defect position.

[0233] Optionally, the device further includes a third determining module, configured to determine the memory feature library in the following manner:

[0234] Acquire multiple images of guide wheels without defects, and obtain corresponding normal samples by identifying the area where the rubber material is located in the images of the guide wheels without defects;

[0235] For each normal sample, extracting feature information corresponding to different positions in the normal sample by the feature extractor;

[0236] The memory feature library is determined according to each piece of feature information extracted.

[0237] Optionally, the feature extractor includes a first feature extractor and a second feature extractor; when the third determination module extracts feature information corresponding to different positions in the normal sample through the feature extractor, it is specifically configured to:

[0238] determining first feature information corresponding to each position in the normal sample by the first feature extractor, and determining second feature information corresponding to each position in the normal sample by the second feature extractor; the number of convolution layers of the first feature extractor is different from the number of convolution layers of the second feature extractor;

[0239] The first feature information and the second feature information corresponding to the same position are fused to obtain feature information corresponding to the position.

[0240] Optionally, when determining the memory feature library according to each extracted feature information, the third determining module is specifically configured to:

[0241] Cluster the extracted feature information;

[0242] The memory feature library is determined according to the clustered plurality of feature information.

[0243] Optionally, when determining the memory feature library according to the clustered plurality of feature information, the third determination module is specifically configured to:

[0244] For each category obtained after clustering, the feature information corresponding to the category is subjected to dimensionality reduction processing;

[0245] Each feature information after the dimensionality reduction process is determined as the feature information in the memory feature library.

[0246] Optionally, when determining the defect information of the guide wheel according to the image to be detected, the second determining module 1402 is specifically configured to:

[0247] Defect information of the guide wheel is determined according to the connected domain in the image to be detected.

[0248] Optionally, when determining the defect information of the guide wheel according to the connected domain in the image to be detected, the second determining module 1402 is specifically configured to:

[0249] Determining the number of connected domains in the image to be detected;

[0250] When the number of the connected domains is greater than a second threshold, it is determined that the guide wheel has a defect.

[0251] Optionally, the device further includes: a second defect location determination module, configured to:

[0252] After determining that the guide wheel has a defect, predicting the circular area where the guide wheel is located based on each connected domain existing in the image to be detected;

[0253] The areas in the circular area except the connected domains are determined as defect locations.

[0254] Optionally, the first determining module 1401 is specifically configured to:

[0255] Identifying the area where the non-hot-melt rubber material is located in the guide wheel image by using a recognition model to obtain an image to be detected;

[0256] Among them, the recognition model is a model obtained by training based on the target data set; the target data set is a data set composed of defect-free guide wheel images, which is used to train the recognition model to identify the area where the rubber material of the guide wheel image in the target training set is located.

[0257] Optionally, the recognition model is a YOLO model; the detection head in the YOLO model includes a detection head for segmentation detection and does not include a detection head for target detection.

[0258] Optionally, the device further includes a processing module configured to:

[0259] After determining the defect information of the guide wheel, entering the defect information into a database;

[0260] And / or, after determining the defect information of the guide wheel, the defect information and the corresponding guide wheel image are sent to a display terminal, so that the auditor can review the defect information based on the guide wheel image.

[0261] Optionally, the method is applied to a server; the vehicle runs on two track beams, and an image acquisition device is provided at a preset position between the two track beams, and the image acquisition device is used to capture images of the vehicle passing by; the server is used to receive the image sent by the image acquisition device and generate an image of the bottom of the vehicle; the guide wheel image is an image of the guide wheel segmented from the image of the bottom of the vehicle.

[0262] Optionally, an approaching vehicle detection device and a leaving vehicle detection device are provided within a preset range of the image acquisition device; the approaching vehicle detection device is used to control the image acquisition device to start capturing images when it detects the arrival of a vehicle; the leaving vehicle detection device is used to control the image acquisition device to stop capturing images when it detects the departure of a vehicle.

[0263] Optionally, the image acquisition device is a line array camera, and the images captured by the line array camera are multiple local images of the bottom of the vehicle when the vehicle is running; the bottom image of the vehicle is an image obtained by splicing the multiple local images.

[0264] The guide wheel defect detection device provided by the embodiment of the present invention can achieve the above-mentioned Figure 2 The guide wheel defect detection method of the embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0265] Figure 15 The hardware structure diagram of a server provided by an embodiment of the present invention is shown in FIG. Figure 15 As shown, the server provided by this embodiment includes: at least one processor 1501 and a memory 1502. The processor 1501 and the memory 1502 are connected via a bus 1503.

[0266] During the specific implementation process, at least one processor 1501 executes the computer-executable instructions stored in the memory 1502, so that the at least one processor 1501 executes the method in the above method embodiment.

[0267] The specific implementation process of the processor 1501 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0268] In the above Figure 15 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0269] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage.

[0270] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0271] An embodiment of the present invention further provides a detection system, comprising:

[0272] An image acquisition device is used to capture multiple partial images of the bottom of the vehicle while the vehicle is running, so as to obtain a vehicle bottom image and a guide wheel image; the vehicle bottom image is an image obtained by stitching together the multiple partial images; the guide wheel image is an image of the guide wheel segmented from the vehicle bottom image;

[0273] An approaching vehicle detection device, configured to control the image acquisition device to start capturing images upon detecting an approaching vehicle;

[0274] A vehicle departure detection device is used to control the image acquisition device to stop capturing images when it detects that the vehicle has left.

[0275] The server is configured to determine an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying an area where a non-hot-melt rubber material is located from the guide wheel image;

[0276] Defect information of the guide wheel is determined based on the image to be detected.

[0277] An embodiment of the present invention further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method of the above method embodiment is implemented.

[0278] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method of the above method embodiment when executed by a processor.

[0279] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. The computer-readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0280] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0281] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0282] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0283] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0284] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A guide wheel defect detection method, characterized in that: The method comprises: Determine an image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image; Defect information of the guide wheel is determined based on the image to be detected.

2. The method according to claim 1, characterized in that Determining defect information of the guide wheel according to the image to be detected includes: Feature information corresponding to at least one position in the image to be detected is extracted by a feature extractor, and defect information of the guide wheel is determined based on the extracted feature information.

3. The method according to claim 2, characterized in that Determining defect information of the guide wheel based on the extracted feature information includes: For each position of feature information corresponding to the position, determine the nearest neighbor feature information corresponding to the feature information stored in the memory feature library; Defect information of the guide wheel is determined based on the feature information corresponding to each position and the nearest neighbor feature information.

4. The method according to claim 3, characterized in that Determining defect information of the guide wheel according to the feature information corresponding to each position and the nearest neighbor feature information includes: For each position, determining difference information between the feature information corresponding to the position and the nearest feature information; According to the difference information corresponding to each position, it is determined that the guide wheel has a defect.

5. The method according to claim 4, characterized in that Determining whether the guide wheel has a defect based on the difference information corresponding to each position includes: For any position, determine the anomaly score of the position based on the difference information corresponding to the position; When the abnormality score of any position is greater than a first threshold, it is determined that the guide wheel has a defect.

6. The method according to claim 5, characterized in that The method further comprises: After determining that the guide wheel has a defect, the position where the abnormal value score is greater than the first threshold is determined as the defect position.

7. The method according to claim 3, characterized in that The memory feature library is determined in the following manner: Acquire multiple images of guide wheels without defects, and obtain corresponding normal samples by identifying the area where the rubber material is located in the images of the guide wheels without defects; For each normal sample, extracting feature information corresponding to different positions in the normal sample by the feature extractor; The memory feature library is determined according to each piece of feature information extracted.

8. The method according to claim 7, characterized in that The feature extractor includes a first feature extractor and a second feature extractor; extracting feature information corresponding to different positions in the normal sample by the feature extractor includes: determining first feature information corresponding to each position in the normal sample by the first feature extractor, and determining second feature information corresponding to each position in the normal sample by the second feature extractor; the number of convolution layers of the first feature extractor is different from the number of convolution layers of the second feature extractor; The first feature information and the second feature information corresponding to the same position are fused to obtain feature information corresponding to the position.

9. The method according to claim 7, characterized in that Determining the memory feature library based on each extracted feature information includes: Cluster the extracted feature information; The memory feature library is determined according to the clustered plurality of feature information.

10. The method according to claim 9, characterized in that Determining the memory feature library according to the clustered plurality of feature information includes: For each category obtained after clustering, the feature information corresponding to the category is subjected to dimensionality reduction processing; Each feature information after the dimensionality reduction process is determined as the feature information in the memory feature library.

11. The method according to claim 1, characterized in that Determining defect information of the guide wheel according to the image to be detected includes: Defect information of the guide wheel is determined according to the connected domain in the image to be detected.

12. The method according to claim 11, characterized in that Determining defect information of the guide wheel according to the connected domain in the image to be detected includes: Determining the number of connected domains in the image to be detected; When the number of the connected domains is greater than a second threshold, it is determined that the guide wheel has a defect.

13. The method according to claim 12, characterized in that The method further comprises: After determining that the guide wheel has a defect, predicting the circular area where the guide wheel is located based on each connected domain existing in the image to be detected; The areas in the circular area except the connected domains are determined as defect locations.

14. The method according to any one of claims 1 to 13, characterized in that Determine the image to be detected corresponding to any guide wheel of the vehicle, including: Identifying the area where the non-hot-melt rubber material is located in the guide wheel image by using a recognition model to obtain an image to be detected; Among them, the recognition model is a model obtained by training based on the target data set; the target data set is a data set composed of defect-free guide wheel images, which is used to train the recognition model to identify the area where the rubber material of the guide wheel image in the target training set is located.

15. The method according to claim 14, characterized in that The recognition model is a YOLO model; the detection head in the YOLO model includes a detection head for segmentation detection and does not include a detection head for target detection.

16. The method according to any one of claims 1 to 13, characterized in that After determining the defect information of the guide wheel, the method further includes: Entering the defect information into a database; And / or, the defect information and the corresponding guide wheel image are sent to a display terminal, so that a reviewer can review the defect information based on the guide wheel image.

17. The method according to any one of claims 1 to 13, characterized in that The method is applied to a server; the vehicle runs on two track beams, and an image acquisition device is provided at a preset position between the two track beams, the image acquisition device being used to capture images of the vehicle as it passes; the server is used to receive images sent by the image acquisition device and generate an image of the vehicle bottom; the guide wheel image is an image of the guide wheel segmented from the vehicle bottom image.

18. The method according to claim 17, characterized in that An approaching vehicle detection device and a leaving vehicle detection device are provided within a preset range of the image acquisition device; the approaching vehicle detection device is used to control the image acquisition device to start capturing images when a vehicle is detected to be approaching; the leaving vehicle detection device is used to control the image acquisition device to stop capturing images when a vehicle is detected to be leaving.

19. The method according to claim 17, wherein The image acquisition device is a line array camera, and the images captured by the line array camera are multiple partial images of the bottom of the vehicle when the vehicle is running; the bottom image of the vehicle is an image obtained by splicing the multiple partial images.

20. A guide wheel defect detection device, characterized in that: The device comprises: The first determination module is used to determine the image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained by identifying the area where the non-hot-melt rubber material is located from the guide wheel image; The second determining module is used to determine the defect information of the guide wheel according to the image to be detected.

21. A server, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 19.

22. A detection system, characterized in that: include: An image acquisition device is used to capture multiple partial images of the bottom of the vehicle while the vehicle is running, so as to obtain a vehicle bottom image and a guide wheel image; the vehicle bottom image is an image obtained by stitching together the multiple partial images; the guide wheel image is an image of the guide wheel segmented from the vehicle bottom image; An approaching vehicle detection device, configured to control the image acquisition device to start capturing images upon detecting an approaching vehicle; A vehicle departure detection device is used to control the image acquisition device to stop capturing images when it detects that the vehicle has left. The server is used to determine the image to be detected corresponding to any guide wheel of the vehicle; the image to be detected is an image obtained after identifying the area where the non-hot-melt rubber material is located from the guide wheel image; and the defect information of the guide wheel is determined based on the image to be detected.

23. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 19 is implemented.

24. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 19 when being executed by a processor.

Images