Guide rail surface defect detection method based on illumination condition and related equipment thereof

Abstract

According to the guide rail surface defect detection method based on the illumination condition and the related equipment thereof provided by the embodiment of the invention, when the defect condition of the target guide rail needs to be detected, the defect condition of the target guide rail can be determined by analyzing the image data of the to-be-detected area of the target guide rail under different illumination sequences; through the illumination equipment, the illumination parameters of the illumination equipment are effectively adjusted in real time in the process of detecting the target guide rail to collect different image data, the defect condition of the target guide rail can be analyzed more comprehensively, the detection cost of the defect condition of the target guide rail is effectively reduced, and compared with an existing technical scheme, the detection efficiency is improved. According to the method provided by the embodiment of the invention, the accuracy of the detection result of the target guide rail can be improved, and the performance is better.

Description

Technical Field

[0001] The present application relates to the technical field of guide rail equipment detection, and in particular to a guide rail surface defect detection method based on lighting conditions and related equipment. Background Art

[0002] In actual applications, guide rails are a key mechanical component that is widely used in various fields. Their surface quality directly affects the performance and service life of the entire equipment. During the production and use of the guide rail surface, various types of defects will appear, such as scratches, pits, corrosion, cracks, etc. The presence of these defects will not only reduce the accuracy of the guide rail, but also cause equipment failure or even shutdown. In recent years, image processing-based methods have gradually become a trend in guide rail surface defect detection. Factors such as the high smoothness, strong reflectivity and complex texture of the guide rail surface lead to the fact that lighting conditions have an important influence on the presentation of guide rail surface defects, which brings challenges to image processing algorithms.

[0003] At present, the lighting conditions for rail surface defect detection mainly rely on manual adjustment and regular maintenance, which not only has low adjustment efficiency and high cost, but also has subjective judgment errors. Since the lighting equipment includes parameters such as the position of the lighting unit, lighting intensity, and lighting color temperature, the inspection personnel need to spend a lot of time to debug the lighting equipment before inspection, and the traditional lighting equipment does not have an adaptive adjustment function, and it is impossible to adjust the lighting parameters in real time during the rail inspection process, resulting in a large number of false positives and false negatives, which reduces the robustness of the detection system and affects the performance and service life of the equipment. Therefore, a lightweight and fast reasoning method for optimizing the lighting conditions for rail surface defect detection is a requirement in the production and use process. Summary of the invention

[0004] The present application aims to solve at least one of the above-mentioned technical defects. In view of this, the present application provides a guide rail surface defect detection method based on lighting conditions and related equipment, which are used to solve the technical defect of difficult guide rail defect detection in the prior art.

[0005] A guide rail surface defect detection method based on illumination conditions, comprising:

[0006] Determine the area to be inspected of the target rail;

[0007] Changing the illumination sequence for the area to be inspected of the target guide rail, and collecting first image data sets of the area to be inspected of the target guide rail under different illumination sequences, wherein the first image data set includes first image data of at least one area to be inspected of the target guide rail corresponding to each set of illumination sequences, wherein the illumination sequence includes the position of an illumination unit, the intensity of illumination, and the color temperature of illumination;

[0008] Preprocessing the first image data set to obtain a second image data set;

[0009] The second image data set is input into a preset defect detection model for detection to obtain a detection result of the target guide rail, wherein the detection result of the target guide rail includes a location where a defect exists in the target guide rail, and the preset defect detection model is trained using the image data set corresponding to the area to be detected of the training guide rail as a training sample and the detection result corresponding to the image data set corresponding to the area to be detected of the training guide rail as a sample label.

[0010] Preferably, the detection result of the target guide rail also includes the probability that the target guide rail presents a defect, and the method further includes:

[0011] Analyzing the inspection results of the to-be-inspected area of ​​the target guide rail, and determining all peaks of the probability of the target guide rail presenting defects included in the inspection results of the target guide rail;

[0012] According to all peak values ​​in the probability of the target guide rail presenting defects, a lighting sequence for re-detecting the defects of the target guide rail is set.

[0013] Preferably, the preprocessing of the first image data set to obtain the second image data set includes:

[0014] Performing a first color space conversion process on each first image data of the first image data set to obtain a first target image data set;

[0015] Performing histogram equalization processing on the image data in the first target image data set to obtain a second target image data set;

[0016] The image data of the second target image data set is subjected to a second color space conversion process to obtain a third target image data set as the second image data set.

[0017] Preferably, the performing histogram equalization processing on the image data in the first target image data set includes:

[0018] Calculate the probability distribution result of each brightness value in the V channel of each image data in the first target image data set, wherein the calculation formula of the probability distribution result of each brightness value in the V channel of each image data in the first target image data set is: p(v k) is the probability distribution function of each brightness value in the V channel of each image data in the first target image data set, nk is the number of pixels with brightness value vk in the V channel of each image data in the first target image data set, and n is the total number of pixels of each image data in the first target image data set;

[0019] Based on the probability distribution result of each brightness value in the V channel of each image data in the first target image data set, the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set is calculated, wherein the calculation formula for calculating the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set is: C(v k ) is the cumulative distribution function of each brightness value in the V channel of each image data in the first target image data set, and i and k are indexes in the calculation process;

[0020] Based on the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set, each brightness value in the V channel of each image data in the first target image data set is mapped to a new equalized brightness value, wherein the mapping relationship between each brightness value in the V channel of each image data in the first target image data set and the equalized brightness value in a mapping relationship therewith is v' k =(F-1)·C(v k ), where v k A new equalized brightness value corresponds to each brightness value in the V channel of each image data in the first target image data set, and F is the maximum brightness value of each image data in the first target image data set.

[0021] Preferably, analyzing the inspection result of the to-be-inspected area of ​​the target guide rail to determine all peaks in the probability of the target guide rail presenting a defect included in the inspection result of the target guide rail comprises:

[0022] Multi-scale analysis of the probability of the target guide rail presenting a defect included in the inspection result of the to-be-inspected area of ​​the target guide rail, so as to smooth the data of the probability of the target guide rail presenting a defect at different scales to generate a multi-scale target signal;

[0023] Performing local peak detection on the target signal at each scale, marking the position of the target rail corresponding to all local maximum values ​​in the target signal at each scale;

[0024] Counting the occurrence frequencies of local peaks at the same position of the target guide rail at different scales to generate a defect peak frequency distribution result of the target guide rail;

[0025] According to the defect peak frequency distribution result of the target guide rail, the peaks that meet the preset conditions at each scale are determined as all the peaks of the probability of the target guide rail presenting a defect.

[0026] Preferably, the preset defect detection model includes an encoder and a decoder;

[0027] Wherein, the encoder is used to extract multi-scale feature information of image data of the object to be detected under different lighting conditions;

[0028] The decoder is used to perform an anomaly score analysis on the multi-scale feature information extracted by the encoder based on a likelihood value anomaly score analysis method, and generate an anomaly map corresponding to the pixel level of the object to be detected; and calculate the false positive data of the generated anomaly map corresponding to the pixel level of the object to be detected, and perform automatic threshold selection and anomaly segmentation processing on it;

[0029] The formula for calculating the anomaly score of the multi-scale feature information extracted by the encoder is: Wherein, AS(i,j) represents the abnormal score of the pixel point with coordinates (i,j); (i,j) represents the pixel position coordinate information in the image data corresponding to the object to be detected extracted by the encoder; represents the white Gaussian embedding generated by the normalized data stream, corresponding to the embedding value of the kth channel of the pixel (i, j) in the image data corresponding to the object to be detected extracted by the encoder at the lth scale, which obeys the standard normal distribution; L represents the total number of scales extracted by the multi-scale features of the image data corresponding to the object to be detected extracted by the encoder; C l Indicates the number of channels at the lth scale;

[0030] The formula for calculating the false positive data of the abnormal map corresponding to the pixel level of the generated object to be detected and performing automatic threshold selection and abnormal segmentation processing on it is:

[0031]

[0032] in, represents the characteristic information of the image; k represents the index in the calculation process; NFA(E) represents the false alarm number function; u(i,j) is a defined variable, that is, a mapping function representing the abnormal score, which has a degree of freedom of C l The chi-square distribution of x 2 (C l ); is the probability of abnormality of pixel (i, j) under the background model H0; is the degree of freedom C lThe cumulative distribution function (CDF) of the chi-square distribution; t is the potential feature The threshold of the sum of squares; N T is the number of all possible connected components.

[0033] Preferably, the calculation formula of the total loss function of the preset defect detection model is: T represents the total loss function of the preset defect detection model; A distillation loss representing anomaly scores of multi-scale feature information of the object to be detected extracted by the encoder of the preset defect detection model; represents false positive data of anomaly map corresponding to the pixel level of the object to be detected generated by the decoder of the preset defect detection model; L CE represents the cross entropy loss; α, β and γ are L CE The weight coefficient of .

[0034] A guide rail surface defect detection device based on illumination conditions, comprising:

[0035] A to-be-inspected area determination unit, used to determine the to-be-inspected area of ​​the target guide rail;

[0036] an illumination sequence setting unit, configured to change an illumination sequence for the area to be detected of the target guide rail, and collect a first image data set of the area to be detected of the target guide rail under different illumination sequences, wherein the first image data set includes first image data of at least one area to be detected of the target guide rail corresponding to each set of illumination sequences, wherein the illumination sequence includes a position of an illumination unit, an intensity of illumination, and a color temperature of illumination;

[0037] A data preprocessing unit, configured to preprocess the first image data set to obtain a second image data set;

[0038] A defect detection unit is used to input the second image data set into a preset defect detection model for detection to obtain a detection result of the target guide rail, wherein the detection result of the target guide rail includes a location where a defect exists in the target guide rail, and the preset defect detection model is trained using the image data set corresponding to the to-be-detected area of ​​the training guide rail as a training sample and the detection result corresponding to the image data set corresponding to the to-be-detected area of ​​the training guide rail as a sample label.

[0039] A guide rail surface defect detection device based on lighting conditions, comprising: one or more processors, and a memory; the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the guide rail surface defect detection method based on lighting conditions as described in any of the above introductions are implemented.

[0040] A readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors implement the steps of the guide rail surface defect detection method based on lighting conditions as described in any of the above introductions.

[0041] It can be seen from the technical solutions introduced above that when it is necessary to detect the defects of the target guide rail, the method provided in the embodiment of the present application can determine the area to be detected of the target guide rail; and by changing the illumination sequence for the area to be detected of the target guide rail, the first image data set of the area to be detected of the target guide rail under different illumination sequences is collected, so that the defects of the target guide rail can be detected by analyzing the image data of the area to be detected of the target guide rail under different illumination conditions, wherein the first image data set may include the first image data of the area to be detected of at least one target guide rail corresponding to each group of illumination sequences, wherein the illumination sequence may include the position of the illumination unit, the intensity of the illumination, and the color temperature of the illumination; since the collected image data There may be some redundant or missing data in the image data. Therefore, in order to improve the analysis efficiency and accuracy, after collecting the first image data set of the area to be inspected of the target guide rail under different lighting sequences, the first image data set can be further preprocessed to obtain a second image data set; the second image data set is input into a preset defect detection model for detection, and the detection result of the target guide rail can be obtained, wherein the detection result of the target guide rail may include the location of the defect in the target guide rail, and the preset defect detection model can be trained using the image data set corresponding to the area to be inspected of the training guide rail as a training sample, and the detection result corresponding to the image data set corresponding to the area to be inspected of the training guide rail as a sample label.

[0042] From the above introduction, it can be seen that when it is necessary to detect the defects of the target guide rail, the method provided in the embodiment of the present application can determine the defects of the target guide rail by analyzing the image data of the target guide rail to be detected under different lighting sequences. Through the lighting equipment, it is effective to adjust the lighting parameters of the lighting equipment in real time during the detection of the target guide rail to collect different image data. The defects of the target guide rail can be analyzed more comprehensively, and the cost of detecting the defects of the target guide rail can be effectively reduced. Compared with the existing technical solutions, the method provided in the embodiment of the present application can improve the accuracy of the detection results of the target guide rail and have better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them, Figure 1 A flow chart of a method for detecting guide rail surface defects based on lighting conditions provided in an embodiment of the present application; Figure 2 A schematic diagram of a lighting device provided in an embodiment of the present application; Figure 3 A schematic diagram of an original image of a straight guide rail provided in an embodiment of the present application under a certain lighting condition; Figure 4 A schematic diagram of analyzing the effect of a guide rail defect provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a defect detection system for the optimal lighting equipment provided in this application; Figure 6 A schematic diagram of the structure of a guide rail surface defect detection device based on lighting conditions according to an embodiment of the present application; Figure 7 This is a hardware structure block diagram of a guide rail surface defect detection device based on lighting conditions disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] In recent years, the application of deep learning technology in various fields has made significant progress. However, since the reasoning process of most defect detection models consumes a lot of computing resources and the reasoning speed is difficult to meet actual needs, it is difficult to deploy on embedded devices, which leads to the difficulty and cost of integrating the lighting condition optimization algorithm on the lighting equipment. Therefore, it is urgent to invent a lightweight and fast reasoning speed method for optimizing the lighting conditions of guide rail surface defect detection to meet the needs of production and use. In view of the fact that most of the current guide rail surface defect detection schemes based on lighting conditions are difficult to adapt to complex and changeable business needs, for this reason, the applicant has studied a guide rail surface defect detection scheme based on lighting conditions. When it is necessary to detect the defect of the target guide rail, the method provided in the embodiment of the present application can determine the defect of the target guide rail by analyzing the image data of the target guide rail to be detected under different lighting sequences. Through the lighting device, it is effectively realized to collect different image data by adjusting the lighting parameters of the lighting device in real time during the detection of the target guide rail. The defect of the target guide rail can be analyzed more comprehensively, and the cost of detecting the defect of the target guide rail can be effectively reduced. Compared with the existing technical solutions, the method provided in the embodiment of the present application can improve the accuracy of the detection results of the target guide rail, and the performance is better. The method provided in the embodiment of the present application can be used in many general or special computing device environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or devices, etc. An embodiment of the present application provides a guide rail surface defect detection method based on lighting conditions. The method can be applied to various guide rail detection systems and can also be applied to various computer terminals or smart terminals. The execution subject can be a processor or server of the computer terminal or smart terminal.

[0046] Combine the following Figure 1 , introduces the process of the guide rail surface defect detection method based on lighting conditions given in the embodiment of the present application, such as Figure 1 As shown, the process can include the following steps:

[0047] Step S101, determining the area to be detected of the target rail.

[0048] Specifically, in the actual application process, in order to improve the detection efficiency and improve the detection effect, it is very necessary to determine the target guide rail's area to be detected when detecting the defects of the target guide rail. Determining the target guide rail's area to be detected can effectively focus on the key parts of the target guide rail and improve the detection efficiency. For example, in the actual application process, different parts of the guide rail are subjected to different stresses, wear degrees, etc. during the working process. For example, in the machine tool guide rail, the area close to the tool processing position is more prone to wear due to the cutting force. By determining the target guide rail's area to be detected, the detection focus can be placed on the key parts of the target guide rail that are prone to problems, avoiding indiscriminate comprehensive detection of the entire target guide rail, thereby saving detection time and cost. For some long-distance guide rails, such as conveyor guide rails on automated production lines, the connection parts at both ends and the middle support parts are more critical. The connection parts may become loose or deformed due to frequent docking and disassembly, and the support parts are subjected to greater pressure. Determining these areas to be detected can enable the detection work to more specifically discover potential hidden dangers. Further, in the actual application process, detection equipment and manpower are limited resources. By determining the area of ​​the target rail to be inspected, it is possible to reasonably allocate inspection equipment and professionals based on the importance of the area and the probability of possible problems.

[0049] For example, for the areas to be inspected with high precision requirements, high-precision testing instruments and experienced technicians can be arranged for inspection, while for relatively less critical areas, more conventional inspection methods can be used, so that the inspection resources can be used to the greatest extent and the inspection efficiency can be improved. Determining the areas to be inspected of the target guide rail can also accurately find problems and effectively ensure the performance of the target guide rail. For example. In actual application, the areas to be inspected of the guide rail are often the parts that best reflect its working status. For example, in the elevator guide rail, the wear of specific areas of the car running track (such as the guide rail position corresponding to the car stop floor) directly affects the running stability and safety of the elevator. By determining these areas to be inspected and conducting detailed inspections, the wear amount, flatness and other parameters of the guide rail can be accurately evaluated, so as to determine whether the guide rail can still meet normal working requirements. For another example, for linear guide rails, the surface roughness and hardness changes in the areas where the slider moves frequently have a key impact on the accuracy and service life of the guide rail. Detecting these areas to be inspected can help determine whether the guide rail has problems such as fatigue wear and corrosion, so that maintenance measures can be taken in time to ensure the performance of the guide rail. In actual applications, many rail failures start from local areas. For example, in the shelf rails of the automated warehousing system, if tiny cracks appear in the area where goods frequently enter and exit and are not discovered in time, over time and with the continuous impact of goods, the cracks may expand, eventually causing the rails to break, causing serious accidents such as shelf collapse. Determining the area to be inspected and conducting regular inspections can detect these potential problems at an early stage, repair or replace parts in advance, effectively prevent the occurrence of failures, and ensure the normal operation of the equipment. Therefore, when it is necessary to analyze whether the target rail has defects, the area to be inspected of the target rail can be determined first, so that the area to be inspected of the target rail can be directly analyzed to understand the status of the target rail.

[0050] In actual application, the area to be detected of the target rail can be determined based on the following methods:

[0051] 1. Determine the area to be inspected of the target rail based on the principle of functional importance:

[0052] (1) Prioritize key working areas: For guide rails, the areas that are directly involved in the working process and play a key role in working accuracy and stability are the focus of consideration. For example, in a machine tool guide rail, when the tool cuts the workpiece, the guide rail area below and around the tool is the key part to ensure machining accuracy because it is directly affected by the cutting force. Any slight deformation or wear in these areas may lead to dimensional errors, poor surface roughness, and other problems in the machined workpiece.

[0053] (2) Focus on the connection and interface areas: The connection parts of the guide rails, such as the joints between different guide rail sections, and the interface positions between the guide rails and other components (such as sliders, brackets, etc.) are also extremely important. Taking the conveyor rails on an automated production line as an example, if the connection points of adjacent guide rails are loose or have height differences, the conveyed items will be bumped or even fall when passing. In elevator guide rails, the contact area between the guide rails and the car guide shoes, its flatness and lubrication directly affect the smoothness and comfort of the car operation.

[0054] 2. Determine the area to be inspected of the target rail according to the principle of vulnerability:

[0055] (1) Focus on high wear areas: Areas prone to wear should be identified as areas to be inspected. This usually includes places where moving parts frequently rub against each other, such as the track section where the slider frequently slides in a linear guide. As the slider repeatedly moves on the guide rail, the friction generated will gradually wear out the surface material of the guide rail, especially in high-speed, high-load movement, the wear rate will be faster. Inspection of these high wear areas is crucial to evaluating the service life and performance degradation of the guide rail.

[0056] (2) Consideration of areas susceptible to corrosion: If the guide rail is in a harsh working environment, such as a humid environment with corrosive gases or liquids, the areas susceptible to corrosion need to be treated with special attention. For example, the guide rails of port cranes operating at the seaside are easily corroded due to the high salt content in the air. In addition, the material conveying guide rails in chemical production workshops may be exposed to various chemical substances. Once the protective layer on the surface is damaged, corrosion will occur at the damaged area, affecting the strength and accuracy of the guide rails.

[0057] 3. Consider the historical fault area principle to determine the target rail area to be inspected:

[0058] (1) Review of frequently failed areas: Pay close attention to the areas where the rails have failed in the past. If a certain section of the rail has repeatedly experienced problems such as excessive wear, deformation, or fatigue cracks, then this area should always be listed as an area to be tested in subsequent inspections. This is because these areas are likely to have inherent weak links due to their own structural characteristics, working conditions, and other factors. Even after repair, the probability of problems occurring again is relatively high.

[0059] (2) Reference of fault areas under similar working conditions: The areas where faults frequently occur under similar working conditions for the same type of guide rails should also be used as references for determining the areas to be inspected. For example, in the same series of machine tool guide rails, if it is found that a specific guide rail area of ​​a certain model of machine tool often has problems when processing a certain material, then the same area of ​​other machine tools of the same model under similar processing conditions should also be inspected as a priority.

[0060] 4. Determine the target rail inspection area based on regulations and standards

[0061] (1) Areas specified by industry regulations: Different industries have corresponding regulations and standards for guide rail inspection. For example, in the elevator industry, there are strict safety standards that specify the inspection area, inspection cycle, and inspection indicators for elevator guide rails. These standards are based on a large number of experiments and practical experience to ensure the safety of elevator operation. Determining the area to be inspected in accordance with these regulations and standards is the basic prerequisite for ensuring that the guide rails meet safety and quality requirements.

[0062] (2) Internal quality standards of the enterprise: In addition to industry standards, enterprises may also formulate stricter or more targeted inspection area specifications based on the characteristics of their own products and quality control requirements. For example, high-end precision instrument manufacturers will have higher requirements for the guide rail inspection area and accuracy of their equipment than the industry standards to ensure high performance and high reliability of their products.

[0063] Step S102 : changing the illumination sequence for the area to be inspected of the target guide rail, and collecting a first image data set of the area to be inspected of the target guide rail under different illumination sequences.

[0064] Specifically, in actual application, different types of defects behave differently under different lighting sequences. For example, when a guide rail with surface cracks is illuminated by oblique side light, an obvious shadow will be formed at the crack. Because the presence of surface cracks will cause differences in refraction and reflection of light at its edge, it will appear as a line with light and dark contrast on the image. For example, in a metal guide rail, when the oblique side light angle is 30°-60°, the depth and direction of the surface cracks of the guide rail can be clearly shown by the shape and length of the shadow. Moreover, for deeper cracks, the color of the shadow will be darker, which helps to intuitively judge the severity of the surface cracks of the guide rail. Under vertical strong light, surface cracks may show a phenomenon of interrupted reflection. Due to the uneven surface inside the crack, the light cannot be reflected regularly at the crack as on a flat surface, resulting in a different reflection intensity in the crack area from the surrounding normal surface. If it is a fine crack, a series of dot-shaped areas of reduced reflection may appear; for wider cracks, it appears as a larger interrupted reflection band. If there are corrosion pits on the rail, they will show a clear outline under strong vertical light. Strong light can enhance the shadow effect at the bottom and edge of the pit, clearly showing the shape, size and depth of the pit.

[0065] For example, on the surface of a guide rail after chemical corrosion, the pits will form dark areas on the image due to the change in the angle of light reflection, while the edges of the pits may appear as bright lines due to the accumulation of light, thus outlining the scope of the pits. When the defective guide rail is illuminated with diffuse light, the overall shape and distribution of the corrosion pits are easier to observe. Diffuse light can evenly illuminate the surface, making the positional relationship and relative size between the pits clear at a glance on the image. However, compared with strong light illumination, the depth information of the pits under diffuse light may not be obvious because diffuse light reduces the shadow effect, making the brightness difference between the inside and outside of the pits smaller. When there are signs of wear on the guide rail, side light illumination is very effective for detecting the signs of wear on the guide rail. When there is wear on the surface of the guide rail, the surface roughness and flatness of the wear area will change. Under side light illumination, the wear area will produce irregular reflections due to the microscopic undulations on its surface. For example, in the slider movement area of ​​the linear guide, the worn surface will appear patches of unevenly reflective areas under side light, and the boundaries of these areas can help determine the scope of wear. Moreover, the intensity and distribution of reflections vary with the degree of wear, and the reflections in severely worn areas will be more chaotic. When backlit, the signs of wear may appear as differences in light transmittance. For some rails made of transparent or translucent materials (such as some plastic rails), the worn parts will appear brighter under backlighting due to the reduced thickness. For opaque metal rails, backlighting can highlight the changes in the surface texture of the worn area, making the scratches and pits caused by wear form a more obvious outline in the image, because the reflected light in the worn area contrasts with the reflected light in the surrounding normal area.

[0066] Therefore, after determining the area to be inspected of the target guide rail, the illumination sequence for the area to be inspected of the target guide rail can be changed, and the first image data set of the area to be inspected of the target guide rail under different illumination sequences can be collected, so that the defect of the target guide rail can be analyzed by analyzing the image data of the target guide rail under different illumination sequence conditions. The first image data set may include the first image data of at least one area to be inspected of the target guide rail under each set of illumination sequences.

[0067] For example, the first image data set may include image data of the target rail's inspection area under oblique lighting conditions and image data under vertical strong light. The lighting sequence represents the parameters of the lighting device, and the lighting sequence may include the position of the lighting unit, the intensity of the lighting, and the color temperature of the lighting. In actual applications, a lighting sequence represents a lighting condition, and the images taken under the lighting condition can be understood as each lighting sequence corresponding to an image data.

[0068] In actual application, a high-precision array camera can be used to collect image data of the target rail. You can refer to using a dome light source as a lighting device. Figure 2 As shown, Figure 2 A lighting device that can be used in an experiment for the method provided in an embodiment of the present application. The lighting device has a plurality of LED lamp bead groups that can be controlled independently. The positions of the lighting units in the lighting sequence described above represent the positions of the corresponding lamp bead groups. The lighting conditions can be changed by controlling the color temperature and intensity of the lamp bead groups at various positions of the lighting device. Therefore, in actual application, the lighting sequence described above may include the position information of all independently controllable lamp beads of the lighting device, the lighting intensity of the LED, and the color temperature.

[0069] In actual application, the process of changing the illumination sequence for the target rail can be as follows:

[0070] According to the actual production, some change sequences are pre-set. During operation, the position of the lamp bead group of the lighting equipment can be changed according to the preset setting change sequence. For example, during the inspection process, multi-angle inspection can be adopted: use side light divided into up, down, left and right (that is, control the light intensity of the light-emitting unit on one side of the dome light source to be greater) or control the light intensity on both sides of the ring light source to be stronger and the light intensity of the dome to be weaker, forming a similar lighting effect of a strip light source. Because the structure of the dome light source is high in the middle and gradually decreases toward the periphery. In the actual application process, the light source can also be controlled to achieve low-angle double side light. For each angle light, such as low-angle side light, a small range of adjustments can be made, for example, the light intensity of the light-emitting unit is changed in turn in a clockwise order. For example, the light intensity can be increased by 10 each time. The purpose of this is to increase the number of analysis samples, so as to facilitate the subsequent use of the automatic multi-scale peak detection algorithm to find all defects.

[0071] By collecting the first image data set of the target rail under different illumination sequences to analyze the defect of the target rail, the defect detection capability can be enhanced and different types of defects can be highlighted. For example, different illumination sequences can make various defects in the target rail to be detected appear in different ways. For example, for tiny cracks on the surface, when oblique side light is used, the cracks will produce obvious shadows in the image, making them easier to identify. For corrosion pits on the surface, vertical strong light can be used to clearly show the depth and contour of the pits. By changing the illumination sequence, the collected first image data set can cover the defect characteristics under various illumination conditions, which is helpful to comprehensively discover and distinguish different types of defects. Under certain illumination conditions, defects may be confused with the normal surface texture of the rail. For example, under uniform diffuse light, some shallow scratches may be difficult to distinguish due to their similar reflection to the surface. However, when a point light source is used to illuminate from a specific angle, the reflection pattern at the scratch will change, making it more prominent in the image. This method of changing the illumination sequence can effectively improve the recognition between defects and normal surfaces and reduce the misjudgment rate. The surface of the target rail may have different textures, such as machining marks, roughness changes, etc. By changing the illumination sequence, the changes of these textures under different illumination can be captured. For example, in a dark light environment, rougher texture areas may absorb more light and appear darker; while in a bright light environment, these textures will be more obvious due to the increase in reflected light. Collecting image data sets under different illumination sequences can provide rich materials for subsequent texture analysis and help to more accurately describe the microscopic features of the rail surface. For rails with certain shapes and spatial structures, different illumination sequences can help reconstruct their three-dimensional features. For example, by taking images from different angles and intensities of illumination, the concave and convex conditions of the rail surface can be inferred by using changes in light and shadow. This illumination-based three-dimensional reconstruction method can provide a more intuitive and accurate basis for evaluating the deformation and wear of the rail, which is especially important for the detection of high-precision rails.

[0072] In actual use, the working environment lighting conditions of the general guide rail are diverse. For example, outdoor guide rail equipment may be exposed to changes in brightness due to direct sunlight or cloud cover; indoor guide rail equipment may have different lighting conditions due to factors such as the layout and power of lighting fixtures. By changing the lighting sequence to collect image data sets, the lighting changes in these actual working environments can be simulated, so that the detection system can better adapt to guide rail detection in different environments and improve the versatility and reliability of the detection results. Understanding the changing rules of guide rail images under different lighting sequences is helpful to distinguish image changes caused by ambient light interference from real guide rail defects in actual detection. For example, when there is flickering or uneven light in the detection environment, if the images under different lighting sequences are not studied, the image anomalies caused by these environmental factors may be mistaken for guide rail defects. By pre-collecting image data sets with different lighting sequences, an effective discrimination method can be established to reduce the adverse effects of ambient light interference on the detection results.

[0073] Step S103: preprocess the first image data set to obtain a second image data set.

[0074] Specifically, in the actual application process, the first image data set collected may contain some redundant image data or unclear image data. For example, in the process of collecting the first image data set, the image may contain noise due to environmental factors (such as unstable lighting, electromagnetic interference, etc.) or the device itself (such as thermal noise of the camera sensor). These noises will make the image blurred or granular spots appear, interfering with the subsequent accurate judgment of the guide rail defects. For example, in some industrial environments, the operation of the motor or the electromagnetic radiation of other equipment may cause salt and pepper noise in the image. Therefore, in order to improve the image quality, the first image data set can be preprocessed to obtain the second image data set. The filtering operations in the preprocessing (such as median filtering, Gaussian filtering, etc.) can effectively reduce these noises, make the image clearer, and highlight the real features of the guide rail surface.

[0075] Furthermore, in the process of collecting the first image data set, the collected original image may have a problem of insufficient contrast, which may make some details of the guide rail, such as tiny wear marks or shallow cracks, difficult to distinguish. By preprocessing the first image data set, such as performing histogram equalization or contrast stretching on the first image data set, the grayscale distribution of the image in the first image data set can be adjusted, so that the contrast of the image in the first image data set is enhanced. For example, for an image with uneven illumination, histogram equalization can redistribute the grayscale values ​​of pixels, so that the difference between the originally darker defect area and the surrounding normal area is more obvious, so that it is easier to detect. Furthermore, the collected first image data set may be collected through different camera settings or in different detection scenarios, and the size of the image may also be different. In the subsequent image analysis process of the collected first image data set, a unified image size is very important. For example, in a defect detection algorithm based on deep learning, the model usually needs to input an image of a fixed size. Through the image scaling operation in the preprocessing, all images can be adjusted to the same size to ensure that the algorithm can run normally and will not make wrong judgments due to differences in image size. In particular, if the images in the first image dataset are color images, these images may use different color spaces (such as RGB, HSV, etc.). In actual applications, different color spaces have different conveniences for analyzing the surface features of the guide rail. By preprocessing the first image dataset, the color spaces of the images in the first image dataset can be unified into a standard color space, which can simplify the subsequent image processing and analysis steps. For example, in some cases, after converting the RGB image to the HSV color space, it may be more intuitive and convenient to detect the discoloration of the guide rail surface (such as color change caused by corrosion). Further, preprocessing the first image dataset can achieve data standardization. This includes normalizing the pixel values ​​to a specific range (such as [0,1] or [-1,1]), which can make the data distribution of different images more reasonable. Doing so helps to improve the performance and stability of some machine learning-based detection algorithms. For example, in classification algorithms such as support vector machines (SVM), standardized data can make the algorithm converge faster and more accurately classify the guide rail defects in the image.

[0076] In actual application, the image after preprocessing the first image data is more conducive to extracting valuable image features. For example, by operating the preprocessed image through an edge detection algorithm (such as Canny edge detection), the edge features of the surface defects of the guide rail can be more clearly extracted. If the original image is of poor quality and has not been preprocessed, the effect of edge detection may be affected, resulting in the inability to accurately extract the edge information of the defect, thereby affecting the judgment of the type and degree of the defect.

[0077] The process of preprocessing the first image data set to obtain the second image data set may include the following steps:

[0078] Step S1031 , performing a first color space conversion process on each first image data set of the first image data set to obtain a first target image data set.

[0079] Specifically, for example, each first image data of the first image data set may be converted from an RGB color space to an HSV color space.

[0080] Step S1032: Perform histogram equalization processing on the image data in the first target image data set to obtain a second target image data set.

[0081] Specifically, the process of performing histogram equalization processing on the image data in the first target image data set may include the following steps:

[0082] Step S10321, calculating the probability distribution result of each brightness value in the V channel of each image data in the first target image data set.

[0083] Specifically, the calculation formula of the probability distribution result of each brightness value in the V channel of each image data in the first target image data set can be shown in the following formula (1):

[0084]

[0085] p(v k ) may be a probability distribution function of each brightness value in the V channel of each image data in the first target image data set;

[0086] n k The brightness value in the V channel of each image data in the first target image data set can be v k The number of pixels;

[0087] n may be the total number of pixels of each image data in the first target image data set;

[0088] Step S10322, based on the probability distribution result of each brightness value in the V channel of each image data in the first target image data set, calculate the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set.

[0089] Specifically, the calculation formula for calculating the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set can be shown in the following formula (2):

[0090]

[0091] Among them, C(v k ) may be a cumulative distribution function of each brightness value in the V channel of each image data in the first target image data set;

[0092] i and k are the indices used in the calculation. For example, if there are F brightness levels, then v k It represents a brightness level, p(v k ) represents the normalized data. For example, if there is a 4×4 image, then n=16, and there are 5 brightness levels, 0 to 4, and k is the index traversing 0 to 4.

[0093] Step S10323, based on the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set, map each brightness value in the V channel of each image data in the first target image data set to a new equalized brightness value.

[0094] The mapping relationship between each brightness value in the V channel of each image data in the first target image data set and the equalized brightness value in a mapping relationship therewith can be shown in the following formula (3):

[0095] v' k =(F-1)·C(v k ) (3)

[0096] Among them, v' k A new equalized brightness value may be corresponding to each brightness value in the V channel of each image data in the first target image data set;

[0097] F may be the maximum value of brightness among the image data in the first target image data set.

[0098] Step S1033: Perform a second color space conversion process on the image data of the second target image data set to obtain a third target image data set as the second image data set.

[0099] Specifically, from the above introduction, it can be seen that the second target image data set is converted from RGB color space to HSV color space. Further, the image data of the second target image data set can be converted from HSV color space to RGB color space, and the third target image data set obtained by the conversion is used as the second image data set.

[0100] Step S104: input the second image data set into a preset defect detection model for detection to obtain a detection result of the target guide rail.

[0101] Specifically, it can be seen from the above introduction that a second image data set with relatively high quality can be obtained by preprocessing the first image data set, and the second image data set includes image data of the inspected area of ​​the target rail under different illumination sequences. Therefore, by analyzing each image data of the second image data set, different types of defects presented by the target rail under different illumination sequences can be understood. Therefore, after obtaining the second image data set, in order to improve the detection efficiency and accuracy of the target rail, the second image data set can be input into a preset defect detection model for detection, so that the detection result of the target rail can be obtained.

[0102] The detection result of the target guide rail may include the location of defects in the target guide rail and probability data of defects in the target guide rail.

[0103] The preset defect detection model can be trained using the image data set corresponding to the area to be inspected of the training rail as training samples and the detection results corresponding to the image data set corresponding to the area to be inspected of the training rail as sample labels.

[0104] For example, in the actual application process, the image data set A corresponding to the training track under different lighting conditions can be collected first, and the image data A corresponding to the training track under different lighting conditions can be preprocessed to obtain the image data set B. Then, a STU-Flow model is constructed as the teacher model based on the sliding window transformation model (Swin Transformer) and the normalizing flow (NormalizingFlow, NF). Then, the constructed teacher model is trained using the preprocessed image data set B corresponding to the training track; the sliding window transformation structure in the created teacher model can perform self-attention calculation; the Swin Transformer structure in the teacher model is a hierarchical structure that can perform multi-scale feature learning; the Swin Transformer structure in the teacher model adopts a hierarchical design similar to CNN, and constructs feature maps of different levels by gradually downsampling. At each stage, it divides the input feature map into different windows and performs self-attention calculations within the window. This hierarchical structure enables the model to effectively learn features of different scales, thereby better adapting to the needs of various visual tasks. For example, in image classification tasks, the model can focus on both global and local features of the image to improve classification accuracy; in target detection tasks, it can better detect targets of different sizes. As the network deepens, the resolution of the feature map gradually decreases and the number of channels gradually increases, which allows the model to capture feature information of different levels of abstraction at different levels. This flexible feature expression capability helps the model better understand and process complex visual data. Among them, the Swin Transformer structure can be used as an encoder for the teacher model to extract multi-scale feature information of image data under different lighting conditions for the training guide; the Normalizing Flow (NF) structure can be used as a decoder for the teacher model, and the multi-scale feature information can be input into the decoder, and anomaly scores are constructed based on likelihood values ​​to generate pixel-level anomaly maps.

[0105] Therefore, the preset defect detection model may include an encoder and a decoder. The encoder may be used to extract multi-scale feature information of image data of the object to be detected under different lighting conditions; the decoder may be used to perform anomaly score analysis on the multi-scale feature information extracted by the encoder based on the likelihood value anomaly score analysis method, and generate an anomaly map corresponding to the pixel level of the object to be detected;

[0106] The formula for calculating the abnormality score of the multi-scale feature information extracted by the encoder can be shown in the following formula (4):

[0107]

[0108] Among them, AS(i,j) can represent the abnormal score of the pixel with coordinates (i,j);

[0109] (i, j) may represent pixel position coordinate information in the image data corresponding to the object to be detected extracted by the encoder;

[0110] It can represent the white Gaussian embedding generated by the normalized data stream (NF), corresponding to the embedding value of the kth channel of the pixel (i, j) in the image data corresponding to the object to be detected extracted by the encoder, which obeys the standard normal distribution;

[0111] L may represent the total number of scales extracted by the encoder from the multi-scale features of the image data corresponding to the object to be detected;

[0112] C l The number of channels that can represent the lth scale;

[0113] After calculating the anomaly score of the multi-scale feature information extracted by the encoder, the false positive data of the anomaly map corresponding to the pixel level of the generated object to be detected can be calculated based on the hypothesis test in the acontrario framework, and automatic threshold selection and anomaly segmentation processing can be performed on it. Among them, the formula for calculating the false positive data of the anomaly map corresponding to the pixel level of the generated object to be detected, and automatically selecting the threshold and anomaly segmentation processing can be shown in the following formulas (5)-(8):

[0114]

[0115]

[0116] in, In formula (8), l represents the scale, r represents the number of pixels in the connected area, and H l ×W l is the image resolution at the lth scale. α1 and β1 are constants, which can be 0.316915 and 4.062570 respectively;

[0117] Among them, C l The number of channels that can represent the lth scale; It can represent the characteristic information of the image; k can represent the index in the calculation process; NFA(E) can represent the false alarm number function; u(i,j) can be a defined variable, which can be specifically represented as a mapping function of the anomaly score, and the variable obeys the degree of freedom C l The chi-square distribution of x 2 (C l ); It can be the probability of abnormality of pixel (i, j) under the background model H0; is the degree of freedom C l The cumulative distribution function (CDF) of the chi-square distribution; t is the potential feature The threshold of the sum of squares; N T The number of all possible connected components that need to be detected in the entire image, that is, the data of the candidate region to be tested, can be obtained by controlling the value of NFA to obtain an unsupervised detection threshold to ensure that no more than a certain number of false positive detections are generated in normal images.

[0118] In actual application, once the NFA value of each connected component is calculated, the last step is to perform anomaly detection under the set threshold. Usually, the threshold used is NFA ≤ 1, that is, those connected components that are not accidental are detected, and these areas are considered abnormal. In actual application, since there are many types of defects on the surface of the guide rail and there are few defect samples, the self-supervised model can be constructed and trained based on positive samples. By calculating the pixel-level anomaly score and performing anomaly segmentation, the detection and location segmentation of various defects can be achieved.

[0119] Through experiments, it was found that the created teacher model can analyze the image data of the inspection area of ​​the training guide rail under different lighting conditions to determine the defect location and presentation probability data of the training guide rail.

[0120] In actual application, SwinTransformer and TinyViT have many differences in performance:

[0121] 1. Accuracy: SwinTransformer is a very powerful visual Transformer model that can achieve excellent performance on various visual tasks. For example, the SwinTransformer model pre-trained on a large-scale dataset can achieve high accuracy on datasets such as ImageNet. Taking Swin-B as an example, it can achieve a high top-1 accuracy on ImageNet, providing a powerful basic model for many downstream tasks. TinyViT also performs well in terms of accuracy. Through techniques such as knowledge distillation, TinyViT can still achieve a high accuracy while greatly reducing the number of parameters. For example, TinyViT with 21M parameters can achieve 84.8% top-1 accuracy on ImageNet-1k, and even 86.5% accuracy at higher resolutions, which is close to or even exceeds some larger Transformer models.

[0122] 2. Parameters and computation: Although Swin Transformer has excellent performance, due to its complex architecture and multi-level design, the number of parameters and computation is relatively large, which makes Swin Transformer face certain challenges when deployed on resource-constrained devices. For example, in mobile devices or edge computing scenarios, higher computing resources and storage resources may be required to run the model.

[0123] TinyViT is a miniaturized visual Transformer model. Its parameters and computational complexity have been carefully optimized and are much smaller than those of the Swin Transformer. This makes TinyViT more applicable in resource-constrained environments, allowing it to run efficiently on devices with limited hardware resources, while also facilitating rapid reasoning and deployment of the model.

[0124] 3. Adaptability to data: After pre-training on large-scale datasets, Swin Transformer can learn rich image features and has good adaptability to large-scale image data. However, for small-scale or field-specific datasets, further fine-tuning or optimization may be required to adapt to the characteristics of the data. TinyViT is designed to be more adaptable to small-scale data. Through techniques such as knowledge distillation, TinyViT can learn common image features from large-scale pre-training models, and can also converge quickly on small-scale data to achieve better performance. This gives TinyViT certain advantages in scenarios where the amount of data is limited.

[0125] 4. Model complexity and scalability: Swin Transformer has a high model complexity, and its architecture design includes multiple stages and modules, such as window attention mechanism, hierarchical structure, etc., which makes the model training and optimization process relatively complex. However, this complex architecture also provides the model with powerful representation capabilities and can handle various complex visual tasks. The TinyViT model has relatively low model complexity, and its structure is relatively simple, easy to train and optimize. At the same time, the design of the TinyViT model also has certain scalability, and can adapt to different task requirements and hardware environments by adjusting the parameters and structure of the model.

[0126] Therefore, based on the advantages presented by TinyViT above, in order to further improve the computing power of the teacher model and improve the detection efficiency of the model, TinyViT (micro visual transformation structure) can be used to replace the Swin Transformer network in the STU-Flow model (i.e., the created teacher model) to construct the TVU-Flow model as the student model. Therefore, the preset defect detection model can also include the TinyViT structure (micro visual transformation structure).

[0127] After building the student model, in order to make the performance of the constructed student model closer to the teacher model, the total loss function can be further constructed, and the response-based knowledge distillation method can be used to further train the student model (TVU-Flow model) by minimizing the output difference between the teacher model (STU-Flow model) and the student model (TVU-Flow model) so that the detection and analysis results of the student model are closer to the expected value. Among them, TinyViT and Swin Transformer are both Transformer architectures, which only play the role of feature extraction in the model. Using TinyViT instead of Swin Transformer can lighten the model, improve the efficiency of reasoning, and reduce the difficulty of deployment on embedded devices.

[0128] Therefore, the calculation formula of the total loss function of the preset defect detection model can be described by the following formula (9):

[0129]

[0130] T can represent the total loss function of the preset defect detection model;

[0131] It can represent the distillation loss of the abnormality score of the multi-scale feature information of the object to be detected extracted by the encoder of the preset defect detection model, that is, In order to calculate the distillation loss of the anomaly score based on the mean square error (MSE) between the pixel-level anomaly scores of the teacher model and the student model introduced above, the calculation formula can be shown as follows (10):

[0132]

[0133] Among them, A t (i, j) and A s (i, j) are the pixel-level anomaly scores of the teacher model and the student model, respectively. H and W represent the height and width of the image, respectively.

[0134] It can represent the false positive data of the abnormality map corresponding to the pixel level of the object to be detected generated by the decoder of the preset defect detection model, that is, is the distillation loss for the number of false positives calculated based on the mean squared error between the number of false positives of the teacher model and the student model introduced above, The calculation formula is shown in (11):

[0135]

[0136] Among them, NFA t (i, j) and NFA s (i, j) represents the number of false positives of the teacher model and the student model at pixel (i, j) introduced above; L CE It can be expressed as cross entropy loss, that is, L CE The cross entropy loss is calculated by generating pseudo labels through self-supervised learning of the teacher model and based on the category probability predicted by the student model; the calculation formula can be shown as follows (12):

[0137]

[0138] in, is the pseudo label generated by self-supervised learning, p s (i, j) is the class prediction probability of the student model at pixel (i, j). α, β, and γ can be weight coefficients, that is, α, β, and γ are the weight coefficients of self-supervised cross entropy loss, anomaly score distillation loss, and false positive number distillation loss, respectively.

[0139] In practical applications, the student model is trained by using positive and negative samples and a response-based knowledge distillation method. The student model achieves performance close to that of the teacher model, but the number of parameters, computational complexity, and storage requirements are greatly reduced, thus reducing the complexity of the model.

[0140] For example, the method provided in the embodiments of the present application is used to analyze Figure 3 The original image data shown can be obtained as Figure 4 The defect effect diagram shown in Figure 4 In the figure, the white area is the detection Figure 3 The location of the defect determined by the original image data shown in the figure is known from experiments. Figure 4 The white area in the middle is indeed the area where the film is flooded in the object to be inspected. Therefore, it can be seen that the method provided in the embodiment of the present application can well determine the specific location of the defect existing in the object to be inspected. In particular, the method provided in the embodiment of the present application can be applied to Figure 5 In the system architecture shown, Figure 5The schematic diagram of the defect detection system structure of the optimal lighting device provided in this application includes a high-precision array camera, an edge computing device, a controller, and a ring lighting device. Among them, the edge computing device is connected to the controller through a high-speed serial port, and sends a lighting sequence to control the ring lighting device to change the lighting conditions; the optimization model of the lighting conditions for the above-mentioned guide rail surface defect detection is deployed on the edge computing device, and the edge computing device is equipped with a storage device for storing the image data and corresponding lighting sequence of each round of optimization. The next round of optimization will overwrite the data of the previous round; after each round of optimization, if an abnormal sample is detected, the best lighting sequence and corresponding image data showing the defect will be output to the next level defect detection system for defect category identification and abnormal segmentation.

[0141] It should be noted that Figure 5 The structural diagram of the defect detection system of the optimal lighting equipment shown is only a system structural diagram of this embodiment. The present application is not limited to this framework, and the next-level defect detection system is not limited to the identification of defect categories and anomaly segmentation; the frequency of optimization in the embodiment can be changed according to actual production requirements.

[0142] Step S105 , analyzing the inspection results of the to-be-inspected area of ​​the target guide rail, and determining all peaks in the probability of the target guide rail presenting defects included in the inspection results of the target guide rail.

[0143] Specifically, from the above introduction, it can be seen that the method provided in the embodiment of the present application can determine the detection result of the area to be inspected of the target guide rail, wherein the detection result of the area to be inspected of the target guide rail may include the probability that the target guide rail presents defects.

[0144] In actual applications, there is a correlation between all peaks in the probability of the target rail presenting defects and the defects presenting the target rail. For example, peaks represent high-probability defect areas. Peaks in the probability distribution of the target rail presenting defects can indicate that the defects are most likely to be detected under these specific lighting sequences. This is because different lighting sequences will highlight the characteristics of the rail surface in different ways. For example, when the lighting angle and intensity are in certain specific combinations, defects such as cracks, wear or corrosion on the rail surface will form a sharp contrast with the normal surface, resulting in peaks in the probability distribution. The lighting sequences corresponding to these peaks can expose defects to the greatest extent, just like shining a flashlight on an object from a specific angle in the dark can most clearly see the flaws on its surface.

[0145] Therefore, after determining the inspection result of the area to be inspected of the target guide rail, the inspection result of the area to be inspected of the target guide rail may be analyzed to determine all peaks in the probability of the target guide rail presenting a defect included in the inspection result of the target guide rail. The process of analyzing the inspection result of the area to be inspected of the target guide rail to determine all peaks in the probability of the target guide rail presenting a defect included in the inspection result of the target guide rail may include the following steps:

[0146] Step S1051 , performing multi-scale analysis on the probability of the target guide rail presenting defects included in the inspection result of the to-be-inspected area of ​​the target guide rail, so as to smooth the data of the probability of the target guide rail presenting defects at different scales and generate a multi-scale target signal.

[0147] For example, in actual application, an automatic multi-scale peak detection algorithm can be used to perform multi-scale analysis on the probability of a target rail presenting a defect included in the detection result of the target rail's inspection area, wherein the automatic multi-scale peak detection algorithm is also called the AMPD algorithm. AMPD (Accelerated Multi-Dimensional Peak Detection) is an accelerated multi-dimensional peak detection algorithm. It is mainly used to efficiently detect peaks in a data set, especially when processing complex multi-dimensional data (such as signal processing, image data, sensor data, and other multi-dimensional data). The algorithm is based on the local characteristics of the data to find the peak. Usually, the size relationship between a data point and its adjacent data points is compared. If a data point has a larger value than the surrounding data points in its local neighborhood (for example, within a certain window range), then this data point may be a peak. It quickly scans the data and reduces unnecessary calculations by cleverly setting parameters such as window size and moving step size, thereby achieving accelerated detection of peaks.

[0148] The algorithm sets a window that is used to determine the local neighborhood of the data point. The choice of window size is critical and depends on the characteristics of the data and the type of peak you want to detect. For example, a smaller window may be used when detecting a narrow peak in the signal, while a larger window may be required for a wider peak. The algorithm then starts at one end of the data and moves the window according to the set step size, analyzing the data within each window. For the central data point in the window, if its value is greater than the values ​​of other data points in the window and meets certain threshold conditions (such as greater than a preset minimum peak height), then this data point is determined to be a potential peak. The setting of the threshold can help filter out some small fluctuations caused by noise and improve the accuracy of peak detection.

[0149] After the initial peak detection, some post-processing steps may be required. For example, multiple adjacent data points that may belong to the same peak can be merged or adjusted to obtain more accurate peak position and intensity information. This is especially important when dealing with noise interference or large data fluctuations. In image edge detection, the edge of the image can be regarded as a peak in brightness or color change. The AMPD algorithm can find the location of the edge by detecting the peak of the image pixel value.

[0150] Step S1052, performing local peak detection on the target signal at each scale, and marking the positions of the target guide rails corresponding to all local maximum values ​​in the target signal at each scale.

[0151] Step S1053, counting the occurrence frequencies of local peaks at the same position of the target guide rail at different scales, and generating a defect peak frequency distribution result of the target guide rail.

[0152] Step S1054, according to the defect peak frequency distribution result of the target guide rail, determine the peaks that meet the preset conditions at each scale as all the peaks of the probability of the target guide rail presenting defects.

[0153] For example, let a signal be x, whose length is N, and the length of the sliding window k1 corresponds to different scales. The window length k1 ranges from 1 to a maximum value h, where That is half the length of the signal.

[0154] For each scale k1, the local maximum is calculated. For each sliding window, if the value of the center point is larger than the values ​​at both ends of the window, the point is considered to be a "local maximum".

[0155] This multi-scale detection process generates a local maximum matrix called the local maximum spectrogram (LMS) through local maximum detection of different window lengths (scales). Each row of this matrix M represents a scale (different sliding window sizes), each column represents a point in the signal, and the element value in the matrix indicates whether the point is a local maximum at that scale.

[0156] By summing the rows of the local maximum spectrum M, we get a vector γ1, in which each element represents the total number of local maxima at a certain scale. Find the scale λ corresponding to the global minimum in the vector γ1. The scale λ indicates that the distribution of local maxima at this scale is the sparsest, which usually corresponds to the "real" peak in the signal. Delete the parts with scales greater than λ from the local maximum spectrum, retain the local maxima below the scale λ, and get a new local maximum matrix. By calculating the standard deviation of the columns of the new matrix, find the columns with a standard deviation of zero. The indexes corresponding to these columns are the peak positions of the signal. This is because at these positions, the signal appears as local maxima at multiple scales, so these positions can be considered to be the true peaks of the signal.

[0157] Step S106 , setting a lighting sequence for re-detecting defects of the target guide rail according to all peak values ​​in the probability of the target guide rail presenting defects.

[0158] Specifically, as can be seen from the above introduction, the method provided in the embodiment of the present application can determine all peaks of the probability of the target guide rail presenting a defect. In actual application, each defect type has its own unique geometry, depth and surface texture changes. By analyzing the illumination sequence corresponding to the peak value of the probability of presenting a defect in the target guide rail, the illumination conditions that are most suitable for displaying a specific defect can be found. For example, for a defect with a tiny pit on the surface, it may be that under a certain oblique illumination sequence, the combination of the shadow and reflection of the pit makes the defect most easily found, and the probability corresponding to this oblique illumination sequence will have a peak. By analyzing all peak values ​​in the probability of the target guide rail presenting a defect, the accuracy and efficiency of re-detection can be effectively improved. For example, the target guide rail can be re-detected using an illumination sequence set according to all peak values ​​in the probability of the target guide rail presenting a defect, so that the target guide rail defect can be more accurately located. Since these illumination sequences are the conditions that are most likely to display defects after analysis, they can reduce misjudgments and missed judgments during the detection process.

[0159] For example, in a complex industrial production environment, there may be interference factors such as oil, dust, etc. on the surface of the guide rail. The appropriate lighting sequence can penetrate or avoid these interferences and focus directly on the defects, so that the inspector or automatic inspection system can quickly and accurately find the location and type of the defect. When re-inspecting the target guide rail, the use of targeted lighting sequences can avoid the use of unnecessary lighting combinations, thereby saving inspection time and energy. If various lighting sequences are used indiscriminately for repeated inspections, it will not only increase the inspection cost, but also may interfere with the judgment of defects due to too much invalid information. The lighting sequence set according to the probability peak can make the inspection work more focused, focusing on the most effective lighting conditions, and improving the inspection efficiency.

[0160] Therefore, after all peaks in the probability that the target guide rail presents defects are determined, a lighting sequence for re-detecting defects of the target guide rail may be set according to all peaks in the probability that the target guide rail presents defects.

[0161] For example, in automobile production workshops, a large number of automated equipment transport automobile parts through rails. The accuracy of these rails is crucial to ensure the smooth operation of the production line. For example, in the automobile body welding workshop, the welding robot moves along the rails to perform welding operations, and the tiny defects of the rails may cause welding position deviations. In the actual application process, through preliminary inspection of the rails, it can be found that when the illumination sequence with a side light angle of about 45° and moderate illumination intensity is used, the probability of the wear defect on the rail surface appears to be peaked. This is because the reflective difference between the worn area and the normal area of ​​the worn rail surface is the largest under this illumination, which can clearly show the traces of wear. When re-inspecting, this illumination sequence is used as the focus. The inspector can more accurately judge the degree and range of wear through visual inspection or by using a machine vision system, such as whether the wear area exceeds the allowable tolerance range, and thus decide whether the rail needs to be repaired or replaced. Using this illumination sequence based on probability peaks for re-inspection can effectively improve the detection accuracy of wear defects, reduce welding quality problems caused by rail wear, and improve the overall quality and efficiency of automobile production.

[0162] For example, in actual application, the safe operation of elevators depends on the good condition of the guide rails. During long-term use, elevator guide rails may have defects such as cracks and corrosion. For example, in some high-rise buildings in coastal areas, the elevator guide rails are subject to corrosion due to humid sea breeze and salt erosion, and the corrosion problem is more prominent. During the inspection process, after preliminary inspection tests, it can be found that for the corrosion defects of elevator guide rails, when vertical strong light is used for irradiation and a small amount of developer is applied to the surface of the guide rail (to enhance the contrast between the corroded area and the normal area), the probability of defect presentation reaches a peak. This is because the strong vertical irradiation can make the outline of the corrosion pit clearer under the action of the developer, and the shadow inside the pit and the reflection of the edge form a sharp contrast. When inspecting again, the maintenance personnel will follow this lighting sequence and operation method, and use tools such as endoscopes to inspect the guide rails in the elevator shaft. This targeted lighting sequence can help maintenance personnel more accurately assess the depth and scope of corrosion and promptly discover potential safety hazards. For some early corrosion defects, protective measures can be taken, such as repainting and anti-corrosion treatment, to extend the service life of the guide rails and ensure the safe operation of the elevator.

[0163] For another example, in an automated warehousing system, the storage and handling of goods are achieved through shelf rails. Shelf rails frequently bear the weight of goods and the impact of handling equipment, and are prone to defects such as deformation and cracks. For example, in large e-commerce warehouses, a large number of packages are sent in and out every day, and the working intensity of shelf rails is very high. During the inspection process, after a preliminary inspection of the shelf rails, it can be found that when multi-angle ring light illumination is used (such as simultaneous illumination from multiple angles such as 30° and 60° from the top and both sides of the rails), and the illumination sequence with gradually increasing illumination intensity, the probability of crack defects on the surface of the rails appears peak. This is because multi-angle ring light can make cracks produce obvious shadow and reflection changes under illumination at different angles, and these changes are more significant as the illumination intensity increases. During the re-inspection, warehouse maintenance personnel use inspection equipment with this lighting setting to regularly inspect the shelf rails. In this way, crack defects in shelf rails can be discovered more promptly, and repairs or replacements can be made before the cracks expand and cause the shelves to collapse, ensuring the safe and efficient operation of the warehousing system.

[0164] It can be seen from the technical solutions introduced above that the method provided by the embodiment of the present application has a powerful multi-scale feature extraction capability, which improves the accuracy of defect detection of the guide rail. The defect detection model of the Transformer architecture used by the method provided by the embodiment of the present application adopts a hierarchical structure in the process of extracting the features of the guide rail. The image blocks (Tokens) of each layer can capture spatial features of different scales. The bottom layer of Tokens focuses on local details, while the high-level Tokens can capture global information, thereby realizing multi-scale feature extraction. Secondly, the defect detection model provided by the embodiment of the present application has an adaptive attention mechanism, which can dynamically adjust the area and scale of its attention according to the different input data, and can adaptively extract features at different levels to ensure that it can effectively pay attention to macroscopic global information and microscopic local details. And it has the characteristics of lightweight and fast reasoning speed, which is convenient for deployment in embedded devices. Based on the pre-trained high-performance teacher model, the student model is distilled for knowledge, which improves the performance of the model while optimizing the model reasoning speed and memory.

[0165] Furthermore, the method provided in the embodiment of the present application can also effectively reduce the detection cost. By using a self-supervised algorithm, there is no need to label a large amount of data. During the experiment, only positive samples are used in the teacher model training process, and a small number of negative samples are used in the knowledge distillation process, which effectively reduces costs and solves the problem of scarce negative samples. In addition, it replaces the traditional lighting condition adjustment method and can adjust the lighting conditions in real time. Furthermore, the method provided in the embodiment of the present application also has the ability to provide optimal lighting conditions for a variety of defects. Based on the model constructed by positive samples, the method of segmenting defects by calculating abnormal scores has the ability to detect a variety of defects, and all peaks in the defect probabilities predicted by the model are located through an automatic multi-scale peak detection algorithm, thereby determining the optimal lighting conditions for a variety of defects.

[0166] It can be seen from this that when it is necessary to detect the defects of the target guide rail, the method provided in the embodiment of the present application can determine the defects of the target guide rail by analyzing the image data of the target guide rail to be detected under different lighting sequences. Through the lighting equipment, it is effective to adjust the lighting parameters of the lighting equipment in real time during the detection of the target guide rail to collect different image data. The defects of the target guide rail can be analyzed more comprehensively, and the cost of detecting the defects of the target guide rail can be effectively reduced. Compared with the existing technical solutions, the method provided in the embodiment of the present application can improve the accuracy of the detection results of the target guide rail and have better performance.

[0167] The following is a description of the guide rail surface defect detection device based on illumination conditions provided in an embodiment of the present application. The guide rail surface defect detection device based on illumination conditions described below and the guide rail surface defect detection method based on illumination conditions described above can be referred to each other. Figure 6 , Figure 6 Schematic diagram of the structure of a guide rail surface defect detection device based on lighting conditions disclosed in an embodiment of the present application. Figure 6 As shown, the guide rail surface defect detection device based on lighting conditions may include:

[0168] The to-be-inspected area determination unit 101 is used to determine the to-be-inspected area of ​​the target guide rail;

[0169] an illumination sequence setting unit 102, configured to change an illumination sequence for the area to be detected of the target guide rail, and collect a first image data set of the area to be detected of the target guide rail under different illumination sequences, wherein the first image data set includes first image data of at least one area to be detected of the target guide rail corresponding to each set of illumination sequences, wherein the illumination sequence includes a position of an illumination unit, an intensity of illumination, and a color temperature of illumination;

[0170] A data preprocessing unit 103, configured to preprocess the first image data set to obtain a second image data set;

[0171] The defect detection unit 104 is used to input the second image data set into a preset defect detection model for detection to obtain a detection result of the target guide rail, wherein the detection result of the target guide rail includes a location where a defect exists in the target guide rail, and the preset defect detection model is trained using the image data set corresponding to the to-be-detected area of ​​the training guide rail as a training sample and the detection result corresponding to the image data set corresponding to the to-be-detected area of ​​the training guide rail as a sample label.

[0172] Among them, the specific processing flow of each unit included in the above-mentioned guide rail surface defect detection device based on lighting conditions can refer to the relevant introduction of the guide rail surface defect detection method based on lighting conditions in the previous text, and will not be repeated here.

[0173] The guide rail surface defect detection device based on illumination conditions provided in the embodiment of the present application can be applied to guide rail surface defect detection equipment based on illumination conditions, such as terminals: mobile phones, computers, etc. Optionally, Figure 7 The hardware structure diagram of the guide rail surface defect detection device based on lighting conditions is shown. Figure 7 , the hardware structure of the guide rail surface defect detection device based on illumination conditions may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4. In the embodiment of the present application, the number of processor 1, communication interface 2, memory 3 and communication bus 4 is at least one, and the processor 1, communication interface 2 and memory 3 communicate with each other through the communication bus 4. The processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiment of the present application; the memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory; wherein the memory stores a program, and the processor can call the program stored in the memory, and the program is used to: implement the various processing flows in the guide rail surface defect detection scheme based on illumination conditions of the aforementioned terminal. The embodiment of the present application also provides a readable storage medium, which can store a program suitable for execution by the processor, and the program is used to: implement the various processing flows of the aforementioned terminal in the guide rail surface defect detection scheme based on illumination conditions.

[0174] Finally, it should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present application. The various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. The various embodiments can be combined with each other. Thus, the present application will not be limited to the embodiments shown herein but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A guide rail surface defect detection method based on lighting conditions, characterized in that: include: Determine the area to be inspected of the target rail; Changing the illumination sequence for the area to be inspected of the target guide rail, and collecting first image data sets of the area to be inspected of the target guide rail under different illumination sequences, wherein the first image data set includes first image data of at least one area to be inspected of the target guide rail corresponding to each set of illumination sequences, wherein the illumination sequence includes the position of an illumination unit, the intensity of illumination, and the color temperature of illumination; Preprocessing the first image data set to obtain a second image data set; The second image data set is input into a preset defect detection model for detection to obtain a detection result of the target guide rail, wherein the detection result of the target guide rail includes a location where a defect exists in the target guide rail, and the preset defect detection model is trained using the image data set corresponding to the area to be detected of the training guide rail as a training sample and the detection result corresponding to the image data set corresponding to the area to be detected of the training guide rail as a sample label.

2. The method according to claim 1, characterized in that The detection result of the target guide rail also includes the probability that the target guide rail presents a defect. The method further includes: Analyzing the inspection results of the to-be-inspected area of ​​the target guide rail, and determining all peaks of the probability of the target guide rail presenting defects included in the inspection results of the target guide rail; According to all peak values ​​in the probability of the target guide rail presenting defects, a lighting sequence for re-detecting the defects of the target guide rail is set.

3. The method according to claim 1, characterized in that The preprocessing of the first image data set to obtain a second image data set includes: Performing a first color space conversion process on each first image data of the first image data set to obtain a first target image data set; Performing histogram equalization processing on the image data in the first target image data set to obtain a second target image data set; The image data of the second target image data set is converted into a second color space to obtain a third target image data set as the second image data set.

4. The method according to claim 3, characterized in that: The performing histogram equalization processing on the image data in the first target image data set includes: Calculate the probability distribution result of each brightness value in the V channel of each image data in the first target image data set, wherein the calculation formula of the probability distribution result of each brightness value in the V channel of each image data in the first target image data set is: p(v k ) is the probability distribution function of each brightness value in the V channel of each image data in the first target image data set, n k The brightness value in the V channel of each image data in the first target image data set is v k The number of pixels of n is the total number of pixels of each image data in the first target image data set; Based on the probability distribution result of each brightness value in the V channel of each image data in the first target image data set, the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set is calculated, wherein the calculation formula for calculating the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set is: C(v k ) is the cumulative distribution function of each brightness value in the V channel of each image data in the first target image data set, and i and k are indexes in the calculation process; Based on the cumulative distribution result of each brightness value in the V channel of each image data in the first target image data set, each brightness value in the V channel of each image data in the first target image data set is mapped to a new equalized brightness value, wherein the mapping relationship between each brightness value in the V channel of each image data in the first target image data set and the equalized brightness value in a mapping relationship therewith is v′ k =(F-1)·C(v k ), where v′ k A new equalized brightness value corresponds to each brightness value in the V channel of each image data in the first target image data set, and F is the maximum brightness value of each image data in the first target image data set.

5. The method according to claim 2, characterized in that: The analyzing the inspection result of the to-be-inspected area of ​​the target guide rail to determine all peaks in the probability of the target guide rail presenting a defect included in the inspection result of the target guide rail includes: Multi-scale analysis of the probability of the target guide rail presenting a defect included in the inspection result of the to-be-inspected area of ​​the target guide rail, so as to smooth the data of the probability of the target guide rail presenting a defect at different scales to generate a multi-scale target signal; Performing local peak detection on the target signal at each scale, marking the position of the target rail corresponding to all local maximum values ​​in the target signal at each scale; Counting the occurrence frequencies of local peaks at the same position of the target guide rail at different scales to generate a defect peak frequency distribution result of the target guide rail; According to the defect peak frequency distribution result of the target guide rail, the peaks that meet the preset conditions at each scale are determined as all the peaks of the probability of the target guide rail presenting a defect.

6. The method according to claim 1, characterized in that The preset defect detection model includes an encoder and a decoder; Wherein, the encoder is used to extract multi-scale feature information of image data of the object to be detected under different lighting conditions; The decoder is used to perform an anomaly score analysis on the multi-scale feature information extracted by the encoder based on a likelihood value anomaly score analysis method, and generate an anomaly map corresponding to the pixel level of the object to be detected; and calculate the false positive data of the generated anomaly map corresponding to the pixel level of the object to be detected, and perform automatic threshold selection and anomaly segmentation processing on it; The formula for calculating the anomaly score of the multi-scale feature information extracted by the encoder is: Wherein, AS(i,j) represents the abnormal score of the pixel point with coordinates (i,j); (i,j) represents the pixel position coordinate information in the image data corresponding to the object to be detected extracted by the encoder; represents the white Gaussian embedding generated by the normalized data stream, corresponding to the embedding value of the kth channel of the pixel (i, j) in the image data corresponding to the object to be detected extracted by the encoder at the lth scale, which obeys the standard normal distribution; L represents the total number of scales extracted by the multi-scale features of the image data corresponding to the object to be detected extracted by the encoder; C l Indicates the number of channels at the lth scale; The formula for calculating the false positive data of the abnormal map corresponding to the pixel level of the generated object to be detected and performing automatic threshold selection and abnormal segmentation processing on it is: in, represents the characteristic information of the image; k represents the index in the calculation process; NFA(E) represents the false alarm number function; u(i,j) is a defined variable, that is, a mapping function representing the abnormal score, which has a degree of freedom of C l The chi-square distribution of x 2 (C l ); is the probability of abnormality of pixel (i, j) under the background model H0; is the degree of freedom C l The cumulative distribution function (CDF) of the chi-square distribution; t is the potential feature The threshold of the sum of squares; N T is the number of all possible connected components.

7. The method according to claim 6, characterized in that The calculation formula of the total loss function of the preset defect detection model is: T represents the total loss function of the preset defect detection model; A distillation loss representing anomaly scores of multi-scale feature information of the object to be detected extracted by the encoder of the preset defect detection model; Represents false positive data of an anomaly map corresponding to the pixel level of the object to be detected generated by the decoder of the preset defect detection model; L CE represents the cross entropy loss; α, β and γ are L CE The weight coefficient of .

8. A guide rail surface defect detection device based on lighting conditions, characterized in that: include: A to-be-inspected area determination unit, used to determine the to-be-inspected area of ​​the target guide rail; an illumination sequence setting unit, configured to change an illumination sequence for the area to be detected of the target guide rail, and collect a first image data set of the area to be detected of the target guide rail under different illumination sequences, wherein the first image data set includes first image data of at least one area to be detected of the target guide rail corresponding to each set of illumination sequences, wherein the illumination sequence includes a position of an illumination unit, an intensity of illumination, and a color temperature of illumination; A data preprocessing unit, configured to preprocess the first image data set to obtain a second image data set; A defect detection unit is used to input the second image data set into a preset defect detection model for detection to obtain a detection result of the target guide rail, wherein the detection result of the target guide rail includes a location where a defect exists in the target guide rail, and the preset defect detection model is trained using the image data set corresponding to the to-be-detected area of ​​the training guide rail as a training sample and the detection result corresponding to the image data set corresponding to the to-be-detected area of ​​the training guide rail as a sample label.

9. A guide rail surface defect detection device based on lighting conditions, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, implement the steps of the guide rail surface defect detection method based on lighting conditions as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors implement the steps of the guide rail surface defect detection method based on lighting conditions as described in any one of claims 1 to 7.

Images