A method and system for detecting rail damage

Abstract

The application provides a rail damage detection method and system, and relates to the technical field of artificial intelligence.In the application, exemplary image data is subjected to key information mining operation to form an image key information description vector, and user identity description data is subjected to feature space mapping operation to form a user identity mapping vector; the user identity mapping vector and the image key information description vector are subjected to merging operation to form a multi-level merging description vector; based on image effectiveness analysis data, the image key information description vector and the multi-level merging description vector, network optimization operation is performed to form a target image analysis network; through the target image analysis network, target image effectiveness analysis data corresponding to the to-be-processed image data is output, and based on the target image effectiveness analysis data, target to-be-processed image data is screened out; and based on the target to-be-processed image data, a target damage detection result is output.Based on the above, the reliability of rail damage detection can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a rail damage detection method and system. BACKGROUND

[0002] There are various ways to detect rail damage. For example, for external damage of the rail, image acquisition and image analysis can be performed to obtain damage detection results. Specifically, a relatively mature artificial intelligence technology can be used to analyze the collected rail image to output the corresponding rail damage result. Artificial intelligence (AI) is the use of digital computers or digital computer-controlled computing simulation, extension and expansion of human intelligence, perception of the environment, acquisition of knowledge and use of knowledge to obtain the best results.

[0003] However, in the prior art, during the detection of rail damage based on rail images, the reliability of rail damage detection is not high. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a rail damage detection method and system to improve the reliability of rail damage detection to some extent.

[0005] To achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

[0006] A rail damage detection method comprises:

[0007] Extracting example image data and image analysis user feature description data, the image analysis user belongs to a user who performs image validity analysis on the example image data, the user feature description data includes user identity description data of the image analysis user and image validity analysis data formed by the image analysis user analyzing the image validity of the example image data, and the example image data is formed by performing damage detection operation on the rail;

[0008] Through an initial image analysis network, the example image data is subjected to a key information mining operation to form an image key information description vector corresponding to the example image data, and the user identity description data of the image analysis user is subjected to a feature space mapping operation through the initial image analysis network to form a user identity mapping vector corresponding to the image analysis user;

[0009] performing a merging operation on the user identity mapping vector to merge into an image key information description vector corresponding to the exemplary image data, to form a corresponding multi-level merged description vector, which simultaneously carries the image key information of the exemplary image data and the user identity key information of the image analysis user;

[0010] performing a network optimization operation on the initial image analysis network based on the image validity analysis data, the image key information description vector, and the multi-level merged description vector, to form a target image analysis network corresponding to the initial image analysis network;

[0011] After obtaining a plurality of to-be-processed image data, performing an analysis operation on each of the to-be-processed image data through the target image analysis network to output target image validity analysis data corresponding to each of the to-be-processed image data, and based on the target image validity analysis data corresponding to each of the to-be-processed image data, screening one to-be-processed image data from the plurality of to-be-processed image data as a target to-be-processed image data, the plurality of to-be-processed image data all belong to a damage detection operation on a to-be-processed rail to form;

[0012] based on the target to-be-processed image data, performing a damage detection operation on the to-be-processed rail to output a target damage detection result corresponding to the to-be-processed rail, the target damage detection result being used to reflect whether the to-be-processed rail has damage and the type of damage.

[0013] In some preferred embodiments, in the above rail damage detection method, the step of performing a key information mining operation on the exemplary image data through the initial image analysis network to form an image key information description vector corresponding to the exemplary image data comprises:

[0014] performing an image segmentation operation on the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data;

[0015] performing a key information mining operation on each of the plurality of image sub-data through the initial image analysis network to output a local key information description vector corresponding to each of the image sub-data, the local key information description vector corresponding to the image sub-data being used to represent the image key information of each exemplary image included in the image sub-data;

[0016] According to the local key information description vector corresponding to each of the image sub-data, a key information description vector corresponding to the example image data is determined, each local key information description vector in the key information description vector is sequentially arranged and combined based on the corresponding image sub-data in the example image data, and the key information description vector is used to represent the image key information of each example image in the example image data.

[0017] In some preferred embodiments, in the above-mentioned rail damage detection method, the example image data includes a plurality of example images, and the step of performing image segmentation on the example image data to form a plurality of image sub-data corresponding to the example image data includes:

[0018] A first image segmentation parameter and a second image segmentation parameter are determined, the first image segmentation parameter is used to represent the interval image frame number between two adjacent image sub-data formed by segmentation, and the second image segmentation parameter is used to represent the image frame number of the example image included in the image sub-data formed by segmentation.

[0019] Based on the first image segmentation parameter and the second image segmentation parameter, the example image data is subjected to image segmentation operation to form a plurality of image sub-data corresponding to the example image data.

[0020] In some preferred embodiments, in the above-mentioned rail damage detection method, each image sub-data corresponding to the local key information description vector is included in the key information description vector of the example image data; and the step of performing merging operation on the user identity mapping vector to merge into the key information description vector corresponding to the example image data to form a corresponding multi-level merging description vector includes:

[0021] The user identity mapping vector and the local key information description vector of each image sub-data are subjected to merging operation respectively to form a merged key information description vector corresponding to each image sub-data.

[0022] Based on the sequence relationship of each image sub-data in the example image data, the merged key information description vector corresponding to each image sub-data is sequentially arranged and combined to form a multi-level merging description vector corresponding to the example image data.

[0023] In some preferred embodiments, in the above-mentioned rail damage detection method, the local key information description vector corresponding to each image sub-data includes an image granularity level description vector of each example image included in the corresponding image sub-data; and the image sub-data to be processed belongs to any one of the image sub-data included in the example image data.

[0024] The step of performing a merging operation on the user identity mapping vector and the local key information description vector of each of the image sub-data respectively to form a merged key information description vector corresponding to each of the image sub-data comprises:

[0025] The step of performing a merging operation on the user identity mapping vector and the image granularity level description vector corresponding to each of the exemplary images respectively to form a local multi-level merged description vector corresponding to each of the exemplary images comprises:

[0026] The step of combining the local multi-level merged description vector corresponding to each of the exemplary images to form the merged key information description vector corresponding to the image sub-data to be processed.

[0027] In some preferred embodiments, in the above rail damage detection method, the step of performing network optimization operation on the initial image analysis network based on the image validity analysis data, the image key information description vector and the multi-level merged description vector to form a target image analysis network corresponding to the initial image analysis network comprises:

[0028] The step of performing validity analysis operation on the exemplary image data by the initial image analysis network based on the image key information description vector and the multi-level merged description vector to output image validity representation data corresponding to the exemplary image data comprises:

[0029] The step of determining a network learning cost index corresponding to the initial image analysis network based on the difference information between the image validity representation data and the image validity analysis data, and performing network optimization operation on the initial image analysis network based on the network learning cost index to form a target image analysis network corresponding to the initial image analysis network.

[0030] In some preferred embodiments, in the above rail damage detection method, the step of performing validity analysis operation on the exemplary image data by the initial image analysis network based on the image key information description vector and the multi-level merged description vector to output image validity representation data corresponding to the exemplary image data comprises:

[0031] The step of performing validity analysis operation on the exemplary image data by the initial image analysis network based on the image key information description vector to output candidate validity representation data corresponding to the exemplary image data comprises:

[0032] The initial image analysis network is used to perform validity analysis on the example image data according to the multi-level merging description vector, so as to output candidate validity representation data corresponding to the example image data;

[0033] Based on the candidate validity representation data and the pending validity representation data, the image validity representation data corresponding to the example image data is analyzed;

[0034] And, the difference information between the image validity representation data and the image validity analysis data is used to determine a network learning cost index corresponding to the initial image analysis network, and the initial image analysis network is optimized based on the network learning cost index to form a target image analysis network corresponding to the initial image analysis network.

[0035] A network learning cost determination rule of the initial image analysis network is determined.

[0036] The image validity representation data and the image validity analysis data are labeled as to-be-processed data of the network learning cost determination rule, and the network learning cost index corresponding to the initial image analysis network is determined. The network learning cost determination rule is used to analyze a typical data set, and the typical data set refers to a set of image validity analysis data of each example image data of a plurality of image analysis users. The network learning cost determination rule includes a first network learning cost determination rule and a second network learning cost determination rule which are different from each other. When the distribution dispersion of the image validity analysis data included in the typical data set is greater than a preset distribution dispersion, the network learning cost index is calculated by the first network learning cost determination rule. When the distribution dispersion of the image validity analysis data included in the typical data set is less than or equal to the preset distribution dispersion, the network learning cost index is calculated by the second network learning cost determination rule.

[0037] The initial image analysis network is optimized based on the network learning cost index to form a target image analysis network corresponding to the initial image analysis network.

[0038] In some preferred embodiments, in the above-mentioned rail damage detection method, the image key information description vector includes a local key information description vector corresponding to each image sub-data, and the step of performing validity analysis on the example image data according to the image key information description vector by the initial image analysis network to output candidate validity representation data corresponding to the example image data includes:

[0039] performing a vector aggregation operation on the local key information description vector corresponding to each of the image sub-data to form a sub-data aggregation description vector corresponding to each of the image sub-data through the initial image analysis network;

[0040] performing an effectiveness analysis operation on the corresponding image sub-data according to the sub-data aggregation description vector corresponding to each of the image sub-data to output local effectiveness representation data corresponding to each of the image sub-data;

[0041] performing a fusion operation on the local effectiveness representation data corresponding to the plurality of image sub-data included in the example image data to output candidate effectiveness representation data corresponding to the example image data.

[0042] In some preferred embodiments, in the above-mentioned rail damage detection method, the step of performing a damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output a target damage detection result corresponding to the to-be-processed rail comprises:

[0043] The target damage detection network formed by performing the network optimization operation performs a damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output a target damage detection result corresponding to the to-be-processed rail, and the target damage detection network is formed by performing a network optimization operation on an initial damage detection network based on typical image data and actual damage state information of a typical rail corresponding to the typical image data.

[0044] The rail damage detection method further comprises:

[0045] After obtaining a plurality of to-be-analyzed image data, each of the to-be-analyzed image data is analyzed through the target image analysis network to output target image effectiveness analysis data corresponding to each of the to-be-analyzed image data, and the plurality of to-be-analyzed image data are all from a rail to be analyzed and subjected to a damage detection operation to form a target damage detection result corresponding to the rail to be analyzed.

[0046] Each of the to-be-analyzed image data is analyzed through the target damage detection network to output an initial damage detection result corresponding to each of the to-be-analyzed image data.

[0047] Based on the target image effectiveness analysis data corresponding to each of the analyzed image data, a fusion operation is performed on the initial damage detection result corresponding to each of the to-be-analyzed image data to output a target damage detection result corresponding to the rail to be analyzed.

[0048] The embodiment of the present application also provides a rail damage detection system, comprising a processor and a memory, the memory is used for storing a computer program, and the processor is used for executing the computer program to realize the rail damage detection method.

[0049] The rail damage detection method and system provided by the embodiment of the present application can perform key information mining operation on exemplary image data to form an image key information description vector, perform feature space mapping operation on user identity description data to form a user identity mapping vector, perform merging operation on the user identity mapping vector and the image key information description vector to form a multi-level merging description vector, perform network optimization operation based on image validity analysis data, the image key information description vector and the multi-level merging description vector to form a target image analysis network, output target image validity analysis data corresponding to to-be-processed image data through the target image analysis network, filter out target to-be-processed image data based on the target image validity analysis data, and output a target damage detection result based on the target to-be-processed image data. Based on the foregoing, since the image validity is determined before the rail damage detection, the basis for the rail damage detection is more reliable, and in addition, since the user feature description data is fused in the network optimization operation, the image validity analysis of the target image analysis network formed is more likely to match the actual application, and therefore, the reliability of the rail damage detection can be improved to a certain extent.

[0050] To make the above objectives, characteristics and advantages of the present application more apparent, the following preferred embodiments are specifically described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A structural block diagram of a rail damage detection system provided by the embodiment of the present application is shown.

[0052] Figure 2 A flowchart of each step included in a rail damage detection method provided by the embodiment of the present application is shown.

[0053] Figure 3 A schematic diagram of each module included in a rail damage detection device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application but not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings herein can be arranged and designed in various different configurations.

[0055] The following detailed description of the embodiments of the application in the drawings provided is not intended to limit the scope of the claimed application, but merely represents the selected embodiments of the application. Based on the embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0056] As shown in the drawings, the embodiments of the application provide a rail damage detection system. The rail damage detection system can include a memory and a processor. Figure 1

[0057] In detail, the memory and the processor are directly or indirectly electrically connected to realize data transmission or interaction. For example, they can be electrically connected through one or more communication buses or signal lines. The memory can store at least one software function module (computer program) in the form of software or firmware. The processor can be used to execute the executable computer program stored in the memory, thereby realizing the rail damage detection method provided by the embodiments of the application.

[0058] It should be understood that in some specific embodiments, the memory can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), etc.

[0059] It should be understood that in some specific embodiments, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0060] ​It should be understood that in some specific embodiments, the rail damage detection system can be a server with data processing capability.

[0061] In combination Figure 2 , the embodiments of the present application also provide a rail damage detection method, which can be applied to the above-mentioned rail damage detection system. The method steps defined by the process related to the rail damage detection method can be implemented by the rail damage detection system.

[0062] The specific process shown in Figure 2 will be described in detail below.

[0063] Step S110, extract the exemplary image data and the user feature description data of the image analysis user.

[0064] In the embodiments of the present application, the rail damage detection system can extract the exemplary image data and the user feature description data of the image analysis user. The image analysis user belongs to the user who performs image validity analysis on the exemplary image data, and the user feature description data includes user identity description data of the image analysis user and image validity analysis data formed by the image analysis user analyzing the image validity of the exemplary image data. The exemplary image data is formed by performing damage detection operation on the rail. In addition, the user identity description data can be user attribute information and user image related information of the image analysis user, the user attribute information can include account, location and other information, and the user image related information can refer to image related operations and behaviors performed by the image analysis user in the past. Moreover, the extracted exemplary image data can be multiple, and multiple exemplary image data can be used to sequentially optimize the initial image analysis network until the preset condition is met.

[0065] Step S120, by the initial image analysis network, perform key information mining operation on the exemplary image data to form image key information description vector corresponding to the exemplary image data, and by the initial image analysis network, perform feature space mapping operation on the user identity description data of the image analysis user to form user identity mapping vector corresponding to the image analysis user.

[0066] In the embodiment of the present application, the rail damage detection system can perform key information mining operation on the example image data through the initial image analysis network to form an image key information description vector corresponding to the example image data, and perform feature space mapping operation on the user identity description data of the image analysis user through the initial image analysis network to form a user identity mapping vector corresponding to the image analysis user. That is, on the one hand, the key information of the example image data is mined, and on the other hand, the user identity description data is converted to be represented in the form of a vector.

[0067] In step S130, the user identity mapping vector is merged into the image key information description vector corresponding to the example image data to form a corresponding multi-level merged description vector.

[0068] In the embodiment of the present application, the rail damage detection system can perform merging operation on the user identity mapping vector to merge into the image key information description vector corresponding to the example image data to form a corresponding multi-level merged description vector. The multi-level merged description vector carries the image key information of the example image data and the user identity key information of the image analysis user at the same time, that is, at least two dimensions of information, one is the information of the example image data itself, and the other is the information of the user analyzing the example image data.

[0069] In step S140, based on the image effectiveness analysis data, the image key information description vector and the multi-level merged description vector, network optimization operation is performed on the initial image analysis network to form a target image analysis network corresponding to the initial image analysis network.

[0070] In the embodiment of the present application, the rail damage detection system can perform network optimization operation on the initial image analysis network based on the image effectiveness analysis data, the image key information description vector and the multi-level merged description vector, such as optimizing and adjusting the included network parameters, to form a target image analysis network corresponding to the initial image analysis network.

[0071] In step S150, after obtaining a plurality of to-be-processed image data, the target image analysis network is used to analyze each of the to-be-processed image data to output target image effectiveness analysis data corresponding to each of the to-be-processed image data, and based on the target image effectiveness analysis data corresponding to each of the to-be-processed image data, one to-be-processed image data is screened out from the plurality of to-be-processed image data as a target to-be-processed image data.

[0072] In the embodiment of the present application, the rail damage detection system can analyze each of the plurality of to-be-processed image data through the target image analysis network after obtaining the plurality of to-be-processed image data, to output target image validity analysis data corresponding to each of the to-be-processed image data, and based on the target image validity analysis data corresponding to each of the to-be-processed image data, filter one to-be-processed image data from the plurality of to-be-processed image data as a target to-be-processed image data, for example, the to-be-processed image data with the highest validity represented by the target image validity analysis data can be filtered out. The plurality of to-be-processed image data are all formed by performing damage detection operation on the to-be-processed rail, that is, by collecting through an image collection device.

[0073] In step S160, based on the target to-be-processed image data, a damage detection operation is performed on the to-be-processed rail to output a target damage detection result corresponding to the to-be-processed rail.

[0074] In the embodiment of the present application, the rail damage detection system can perform a damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output a target damage detection result corresponding to the to-be-processed rail. The target damage detection result is used to reflect whether the to-be-processed rail has damage and the type of damage, and can also include the degree of damage, etc.

[0075] Based on the foregoing, since the rail damage detection is performed based on the determination of image validity, the basis for performing the rail damage detection is more reliable. In addition, since the user feature description data is fused in the network optimization operation, the image validity analysis of the formed target image analysis network is more easily matched with actual application, so that the reliability of the rail damage detection can be improved to a certain extent, thereby improving the deficiencies in the prior art.

[0076] It should be understood that in some specific embodiments, step S120 in the above implementation, that is, the step of performing key information mining operation on the exemplary image data through the initial image analysis network to form an image key information description vector corresponding to the exemplary image data, can further include the following implementation process:

[0077] The exemplary image data is subjected to image segmentation operation to form a plurality of image sub-data corresponding to the exemplary image data. For example, the exemplary image data can include a plurality of (frame) continuous exemplary images, so that the plurality of exemplary images can be subjected to segmentation operation (segmentation in time dimension) to form a plurality of image sub-data, each of which can include one or more exemplary images.

[0078] Each of the plurality of image sub-data is subjected to a key information mining operation by the initial image analysis network to output a local key information description vector corresponding to each of the image sub-data, the local key information description vector corresponding to each of the image sub-data is used to represent the image key information of each exemplary image included in the image sub-data, for example, each of the image sub-data can be subjected to a convolution operation by a convolution unit to form a local key information description vector corresponding to each of the image sub-data, in addition, for each of the image sub-data, each exemplary image included in the image sub-data can be subjected to a convolution operation, and then the results of the convolution operation can be aggregated, such as cascading combination and the like, to obtain a local key information description vector;

[0079] According to the local key information description vector corresponding to each of the image sub-data, the image key information description vector corresponding to the exemplary image data is determined, each local key information description vector in the image key information description vector is sequentially arranged and combined based on the order of the corresponding image sub-data in the exemplary image data, the image key information description vector is used to represent the image key information of each exemplary image in the exemplary image data, for example, the image key information description vector can be {the local key information description vector corresponding to the image sub-data 1, the local key information description vector corresponding to the image sub-data 2, the local key information description vector corresponding to the image sub-data 3... the local key information description vector corresponding to the image sub-data n}.

[0080] It should be understood that in some specific embodiments, the exemplary image data includes a plurality of exemplary images, based on this, the step of performing image segmentation operation on the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data can further include the following described implementation process:

[0081] The first image segmentation parameter and the second image segmentation parameter are determined, the first image segmentation parameter is used to represent the interval image frame number between two adjacent image sub-data formed by segmentation, and the second image segmentation parameter is used to represent the image frame number of the exemplary image included in the image sub-data formed by segmentation, so that the frame number of the exemplary image included in each image sub-data is consistent;

[0082] Based on the first image segmentation parameter and the second image segmentation parameter, the image segmentation operation is performed on the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data, the first image segmentation parameter and the second image segmentation parameter can be configured according to actual needs, which is not specifically limited and described here.

[0083] It should be understood that in some specific embodiments, the image key information description vector can include a local key information description vector corresponding to each image sub-data, and based on this, the step S130 in the above implementation, i.e., the step of performing a merging operation on the user identity mapping vector to merge into the image key information description vector corresponding to the exemplary image data, to form a corresponding multi-level merging description vector, can further include the implementation process described as follows:

[0084] Perform a merging operation on the user identity mapping vector and the local key information description vector of each image sub-data, respectively, to form a merging key information description vector corresponding to each image sub-data, for example, the user identity mapping vector and the local key information description vector of image sub-data 1 can be merged to form a merging key information description vector corresponding to image sub-data 1, the user identity mapping vector and the local key information description vector of image sub-data 2 can be merged to form a merging key information description vector corresponding to image sub-data 2, and the user identity mapping vector and the local key information description vector of image sub-data 3 can be merged to form a merging key information description vector corresponding to image sub-data 3.

[0085] Based on the order relationship of each image sub-data in the exemplary image data, the merging key information description vector corresponding to each image sub-data is arranged in order to form a multi-level merging description vector corresponding to the exemplary image data, for example, the multi-level merging description vector can be {the merging key information description vector corresponding to image sub-data 1, the merging key information description vector corresponding to image sub-data 2, and the merging key information description vector corresponding to image sub-data 3}.

[0086] It should be understood that in some specific embodiments, the local key information description vector corresponding to each image sub-data includes image granularity level description vectors of each exemplary image included in the corresponding image sub-data (the image granularity level description vectors can be formed by performing a key information mining operation on the exemplary image). The image sub-data to be processed belongs to any one (frame) image sub-data included in the exemplary image data, and based on this, the step of performing a merging operation on the user identity mapping vector and the local key information description vector of each image sub-data, respectively, to form a merging key information description vector corresponding to each image sub-data, can further include the implementation process described as follows:

[0087] The user identity mapping vector and the image granularity level description vector corresponding to each example image are merged to form a local multi-level merged description vector corresponding to each example image. For example, the user identity mapping vector and the image granularity level description vector corresponding to example image 1 are merged to form a local multi-level merged description vector corresponding to example image 1. The user identity mapping vector and the image granularity level description vector corresponding to example image 2 are merged to form a local multi-level merged description vector corresponding to example image 2. The user identity mapping vector and the image granularity level description vector corresponding to example image 3 are merged to form a local multi-level merged description vector corresponding to example image 3.

[0088] The local multi-level merged description vectors corresponding to each example image are combined to form a merged key information description vector corresponding to the image sub-data to be processed. For example, the merged key information description vector can be {local multi-level merged description vector corresponding to example image 1, local multi-level merged description vector corresponding to example image 2, local multi-level merged description vector corresponding to example image 3}.

[0089] It should be understood that in some specific embodiments, the step of merging the user identity mapping vector and the image granularity level description vector corresponding to each example image to form a local multi-level merged description vector corresponding to each example image can further include the following implementation process:

[0090] The image granularity level description vector is weighted based on a first weighting vector to output a corresponding first weighted fusion vector. The user identity mapping vector is weighted based on a second weighting vector and a third weighting vector to output a corresponding second weighted fusion vector and a third weighted fusion vector, respectively. The first weighting vector, the second weighting vector, and the third weighting vector belong to the network parameters of the corresponding neural network. The product of the transpose of the first weighted fusion vector and the second weighted fusion vector is calculated. The third weighting vector is multiplied based on the product or the positive correlation value of the product to output a first-level similar fusion vector.

[0091] filtering the image granularity level description vectors by a plurality of filter units with different sizes to form a plurality of first filtered description vectors, and filtering the user identity mapping vectors by the plurality of filter units to form a plurality of second filtered vectors;

[0092] for each filter unit, weighting the first filtered description vector corresponding to the filter unit based on a first weighting vector to output a corresponding first weighted fusion vector, weighting the second filtered vector corresponding to the filter unit based on a second weighting vector and a third weighting vector to output a corresponding second weighted fusion vector and a third weighted fusion vector, calculating a product between the first weighted fusion vector and a transposed vector of the second weighting vector, and performing a multiplication fusion operation on the third weighting vector based on the product or a positive correlation value of the product to output a corresponding second level similarity fusion vector;

[0093] performing a mean value calculation on the first level similarity fusion vector and the second level similarity fusion vector corresponding to each filter unit to output a corresponding local multi-level merged description vector.

[0094] It should be understood that in some embodiments, the step of performing a mean value calculation on the first level similarity fusion vector and the second level similarity fusion vector corresponding to each filter unit to output a corresponding local multi-level merged description vector can further include the following implementation process:

[0095] performing a mean value calculation on the first level similarity fusion vector and the second level similarity fusion vector corresponding to each filter unit to output a corresponding local multi-level merged description vector, and outputting a corresponding mean value description vector (in other embodiments, the mean value description vector can also be directly used as the corresponding local multi-level merged description vector);

[0096] obtaining rail detection audio data corresponding to the to-be-processed image sub-data, the rail detection audio data belonging to audio data formed by audio collection after hitting a corresponding rail, and each example image included in the to-be-processed image sub-data corresponding to the rail detection audio data having a collection time consistent with a collection time of the rail detection audio data;

[0097] performing a key information mining operation on the rail detection audio data to output an audio key information description vector corresponding to the rail detection audio data;

[0098] The audio key information description vector and the mean description vector are combined to form a corresponding local multi-level combined description vector. The combination operation can refer to the process of mapping the user identity vector to the image granularity level description vector to form the mean description vector described above, and will not be described again.

[0099] It should be understood that in some specific embodiments, the step S140 in the above implementation, i.e., the step of performing network optimization operation on the initial image analysis network based on the image validity analysis data, the image key information description vector and the multi-level combined description vector to form the target image analysis network corresponding to the initial image analysis network, can further include the implementation process described as follows:

[0100] The exemplary image data is analyzed for validity by the initial image analysis network according to the image key information description vector and the multi-level combined description vector to output image validity representation data corresponding to the exemplary image data. The image validity representation data and the image validity analysis data are both used to represent the validity of the exemplary image data, but the determination manner is not consistent.

[0101] According to the difference information between the image validity representation data and the image validity analysis data, a network learning cost index corresponding to the initial image analysis network is determined. For example, the network learning cost index can have a positive correlation with the difference information. Based on the network learning cost index, the initial image analysis network is optimized to form a target image analysis network corresponding to the initial image analysis network.

[0102] It should be understood that in some specific embodiments, the step of analyzing the exemplary image data for validity by the initial image analysis network according to the image key information description vector and the multi-level combined description vector to output image validity representation data corresponding to the exemplary image data can further include the implementation process described as follows:

[0103] The exemplary image data is analyzed for validity by the initial image analysis network according to the image key information description vector to output candidate validity representation data corresponding to the exemplary image data. For example, the image key information description vector can be processed by a first output unit included in the initial image analysis network to output candidate validity representation data. The first output unit can perform full connection and activation processing.

[0104] The initial image analysis network is used to perform validity analysis on the example image data according to the multi-level merged description vector, so as to output the pending validity representation data corresponding to the example image data. For example, the first output unit can be used to process the multi-level merged description vector to obtain the pending validity representation data, or the second output unit included in the initial image analysis network can be used to process the multi-level merged description vector to output the pending validity representation data.

[0105] Based on the candidate validity representation data and the pending validity representation data, the image validity representation data corresponding to the example image data is analyzed. For example, the average or weighted sum of the validity values represented by the candidate validity representation data and the pending validity representation data can be calculated. When the weighted sum is calculated, the weighting coefficient corresponding to the pending validity representation data can be greater than the weighting coefficient corresponding to the candidate validity representation data.

[0106] It should be understood that in some specific embodiments, the image key information description vector includes a local key information description vector corresponding to each image sub-data. Based on this, the step of performing validity analysis on the example image data according to the image key information description vector by the initial image analysis network to output the candidate validity representation data corresponding to the example image data can further include the following implementation process:

[0107] The initial image analysis network is used to perform validity analysis on the example image data according to the multi-level merged description vector, so as to output the pending validity representation data corresponding to the example image data. For example, the first output unit can be used to process the multi-level merged description vector to obtain the pending validity representation data, or the second output unit included in the initial image analysis network can be used to process the multi-level merged description vector to output the pending validity representation data.

[0108] The initial image analysis network is used to perform validity analysis on the example image data according to the multi-level merged description vector, so as to output the pending validity representation data corresponding to the example image data. For example, the first output unit can be used to process the multi-level merged description vector to obtain the pending validity representation data, or the second output unit included in the initial image analysis network can be used to process the multi-level merged description vector to output the pending validity representation data.

[0109] The initial image analysis network is used to perform validity analysis on the example image data according to the multi-level merged description vector, so as to output the pending validity representation data corresponding to the example image data. For example, the first output unit can be used to process the multi-level merged description vector to obtain the pending validity representation data, or the second output unit included in the initial image analysis network can be used to process the multi-level merged description vector to output the pending validity representation data.

[0110] It should be understood that, in some specific embodiments, the step of determining the network learning cost index corresponding to the initial image analysis network according to the difference information between the image effectiveness characterization data and the image effectiveness analysis data, and performing network optimization operation on the initial image analysis network based on the network learning cost index to form the target image analysis network corresponding to the initial image analysis network, can further include the following implementation process:

[0111] Determining a network learning cost determination rule of the initial image analysis network;

[0112] Labeling the image effectiveness characterization data and the image effectiveness analysis data as to-be-processed data of the network learning cost determination rule to determine the network learning cost index corresponding to the initial image analysis network, wherein the network learning cost determination rule performs analysis operation on a typical data set, the typical data set refers to a set of image effectiveness analysis data of each exemplary image data by multiple image analysis users; the network learning cost determination rule includes mutually different first network learning cost determination rule and second network learning cost determination rule, when the distribution dispersion of the image effectiveness analysis data included in the typical data set is greater than a preset distribution dispersion, the network learning cost index is calculated by the first network learning cost determination rule, when the distribution dispersion of the image effectiveness analysis data included in the typical data set is less than or equal to the preset distribution dispersion, the network learning cost index is calculated by the second network learning cost determination rule, and the specific value of the preset distribution dispersion is not limited and can be configured according to actual needs;

[0113] Performing network optimization operation on the initial image analysis network based on the network learning cost index to form the target image analysis network corresponding to the initial image analysis network, and based on this, since the determination rule of the network learning cost index is related to the distribution dispersion of the image effectiveness analysis data, the focus of the target image analysis network formed based on the network learning cost index is more extensive, rather than concentrated in a few directions.

[0114] It should be understood that, in some specific embodiments, the step of calculating the distribution dispersion can further include the following implementation process:

[0115] For a set of image effectiveness analysis data of each exemplary image data by multiple image analysis users, performing deduplication operation on the image effectiveness analysis data included in the set of image effectiveness analysis data to form a sub-set of image effectiveness analysis data;

[0116] determining, for each of the image effectiveness analysis data in the image effectiveness analysis data subset, a number of identical image effectiveness analysis data that the image effectiveness analysis data has in the image effectiveness analysis data set, to obtain a corresponding target data number;

[0117] performing a mean value calculation on the target data number corresponding to each of the image effectiveness analysis data in the image effectiveness analysis data subset, to output a corresponding mean data number, and performing a mean value calculation on an absolute difference value between the target data number corresponding to each of the image effectiveness analysis data and the mean data number, to obtain a corresponding distribution dispersion.

[0118] In some specific embodiments, the step of marking the image effectiveness representation data and the image effectiveness analysis data as to-be-processed data of the network learning cost determination rule to determine the network learning cost indicator corresponding to the initial image analysis network can further include the following implementation process:

[0119] When the distribution dispersion of the image effectiveness analysis data included in the typical data set is less than or equal to the preset distribution dispersion, performing a sum-of-squares calculation on the image effectiveness representation data and the corresponding image effectiveness analysis data included in the typical data set, to output a corresponding network learning cost indicator.

[0120] In some specific embodiments, the step of marking the image effectiveness representation data and the image effectiveness analysis data as to-be-processed data of the network learning cost determination rule to determine the network learning cost indicator corresponding to the initial image analysis network can further include the following implementation process:

[0121] When the distribution dispersion of the image effectiveness analysis data included in the typical data set is greater than the preset distribution dispersion, calculating a mean value of the image effectiveness analysis data included in the typical data set to obtain a first mean value, and calculating a mean value of the corresponding image effectiveness representation data to obtain a second mean value, then calculating a square value of a difference value between the first mean value and the second mean value to obtain a first square value, and calculating a ratio between the first square value and a predetermined adjustment parameter, which can be configured according to actual needs, and performing an exponential operation on a negative correlation value of the ratio to output a first exponential operation value, for example, a sum value between the ratio and the negative correlation value is equal to a preset value, such as 0, 1, 2, 3, etc.

[0122] respectively calculating square values of differences between each image validity analysis data included in the typical data set and corresponding image validity representation data corresponding to the image validity analysis data, to obtain corresponding second square values, respectively calculating ratios between each second square value and the adjustment parameter, and performing exponential operation on a negative correlation value of the ratio to output corresponding second exponential operation values, and calculating a sum value of each second exponential operation value to obtain a target exponential operation value;

[0123] calculating a ratio between the first exponential operation value and the target exponential operation value, performing logarithm operation on the ratio to obtain a corresponding logarithm result value, and finally determining a network learning cost index corresponding to the initial image analysis network based on the logarithm result value, the network learning cost index and the logarithm result value having a negative correlation corresponding relationship, such as a sum value between the network learning cost index and the logarithm result value being equal to a preset value.

[0124] It should be understood that in some specific embodiments, the step S160 in the above implementation, that is, the step of performing damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output a target damage detection result corresponding to the to-be-processed rail, can further include the implementation process described below:

[0125] The target damage detection network formed by the network optimization operation performs damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output a target damage detection result corresponding to the to-be-processed rail, and the target damage detection network is formed by performing network optimization operation on an initial damage detection network based on typical image data and actual damage state information of a typical rail corresponding to the typical image data, for example, the initial damage detection network can perform rail damage analysis on the typical image data to output corresponding predicted damage state information, and then the network parameters included in the initial damage detection network can be updated and optimized based on a difference between the predicted damage state information and the actual damage state information.

[0126] It should be understood that in some specific embodiments, the rail damage detection method can further include the implementation process described below:

[0127] After obtaining a plurality of to-be-analyzed image data, each of the to-be-analyzed image data is analyzed by the target image analysis network to output target image validity analysis data corresponding to each of the to-be-analyzed image data, and the plurality of to-be-analyzed image data are obtained by performing damage detection operation on a to-be-analyzed rail to form a target rail;

[0128] The target damage detection network is used for performing analysis operations on each of the image data to be analyzed to output an initial damage detection result corresponding to each of the image data to be analyzed.

[0129] The target damage detection result corresponding to the steel rail to be analyzed is output by performing fusion operations on the initial damage detection result corresponding to each of the image data to be analyzed based on the target image validity analysis data corresponding to each of the image data to be analyzed. For example, the target image validity analysis data can be used as an importance of the corresponding initial damage detection result. In this way, the target damage detection result can be obtained by fusing each of the initial damage detection results based on the importance.

[0130] In combination Figure 3 The embodiments of the present application also provide a steel rail damage detection device which can be applied to the above-mentioned steel rail damage detection system. The steel rail damage detection device can include:

[0131] An exemplary data extraction module is configured to extract exemplary image data and user feature description data of an image analysis user. The image analysis user belongs to a user who performs image validity analysis on the exemplary image data. The user feature description data includes user identity description data of the image analysis user and image validity analysis data formed by the image analysis user analyzing the image validity of the exemplary image data. The exemplary image data is formed by performing damage detection operations on a steel rail.

[0132] A key information mining module is configured to perform key information mining operations on the exemplary image data by an initial image analysis network to form an image key information description vector corresponding to the exemplary image data. The user identity description data of the image analysis user is subjected to feature space mapping operations by the initial image analysis network to form a user identity mapping vector corresponding to the image analysis user.

[0133] A vector merging module is configured to perform merging operations on the user identity mapping vector to merge into the image key information description vector corresponding to the exemplary image data to form a corresponding multi-level merged description vector. The multi-level merged description vector simultaneously carries image key information of the exemplary image data and user identity key information of the image analysis user.

[0134] A network optimization module is configured to perform network optimization operations on the initial image analysis network based on the image validity analysis data, the image key information description vector and the multi-level merged description vector to form a target image analysis network corresponding to the initial image analysis network.

[0135] The image data screening module is configured to, after obtaining a plurality of image data to be processed, analyze each of the image data to be processed by using the target image analysis network to output target image validity analysis data corresponding to each of the image data to be processed, and screen one of the image data to be processed as target image data to be processed based on the target image validity analysis data corresponding to each of the image data to be processed, wherein the plurality of image data to be processed are all obtained by performing damage detection on a steel rail to be processed;

[0136] The steel rail damage detection module is configured to perform damage detection on the steel rail to be processed based on the target image data to be processed to output a target damage detection result corresponding to the steel rail to be processed, wherein the target damage detection result is used to reflect whether the steel rail to be processed has damage and a type of the damage.

[0137] In summary, the steel rail damage detection method and system provided by the present application can first perform key information mining on exemplary image data to form an image key information description vector, perform feature space mapping on user identity description data to form a user identity mapping vector, perform merging on the user identity mapping vector and the image key information description vector to form a multi-level merged description vector, perform network optimization based on the image validity analysis data, the image key information description vector and the multi-level merged description vector to form a target image analysis network, output target image validity analysis data corresponding to image data to be processed by using the target image analysis network, screen target image data to be processed based on the target image validity analysis data, and output a target damage detection result based on the target image data to be processed. Based on the foregoing, the basis for performing damage detection on the steel rail is more reliable because the image validity is determined before the damage detection is performed. In addition, the image validity analysis of the target image analysis network formed in the network optimization process is more easily matched with actual application because the user feature description data is fused, and therefore the reliability of the damage detection on the steel rail can be improved to a certain extent.

[0138] The above only describes preferred embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made by those skilled in the art based on the spirit and principles of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of rail damage detection, characterized by, The method comprises the following steps: extracting user feature description data of an exemplary image data and an image analysis user, the image analysis user belongs to a user who performs image validity analysis on the exemplary image data, the user feature description data comprises user identity description data of the image analysis user and image validity analysis data formed by the image analysis user analyzing image validity of the exemplary image data, and the exemplary image data is formed by performing a damage detection operation on a steel rail; performing a key information mining operation on the exemplary image data by an initial image analysis network to form an image key information description vector corresponding to the exemplary image data, and performing a feature space mapping operation on the user identity description data of the image analysis user by the initial image analysis network to form a user identity mapping vector corresponding to the image analysis user; performing a merging operation on the user identity mapping vector to merge into the image key information description vector corresponding to the exemplary image data to form a corresponding multi-level merged description vector, the multi-level merged description vector simultaneously carries image key information of the exemplary image data and user identity key information of the image analysis user; performing a network optimization operation on the initial image analysis network based on the image validity analysis data, the image key information description vector and the multi-level merged description vector to form a target image analysis network corresponding to the initial image analysis network; after obtaining a plurality of to-be-processed image data, performing an analysis operation on each of the to-be-processed image data by the target image analysis network to output target image validity analysis data corresponding to each of the to-be-processed image data, and based on the target image validity analysis data corresponding to each of the to-be-processed image data, screening one to-be-processed image data from the plurality of to-be-processed image data as a target to-be-processed image data, the plurality of to-be-processed image data all belong to a damage detection operation performed on a to-be-processed steel rail to form; based on the target to-be-processed image data, performing a damage detection operation on the to-be-processed steel rail to output a target damage detection result corresponding to the to-be-processed steel rail, the target damage detection result is used to reflect whether the to-be-processed steel rail has damage and the type of the damage; the step of performing a key information mining operation on the exemplary image data by an initial image analysis network to form an image key information description vector corresponding to the exemplary image data comprises: performing an image segmentation operation on the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data; performing a key information mining operation on each of the plurality of image sub-data by an initial image analysis network to output a local key information description vector corresponding to each of the image sub-data, the local key information description vector corresponding to the image sub-data is used to represent image key information of each exemplary image included in the image sub-data; Determine the image key information description vector corresponding to the exemplary image data according to the local key information description vector corresponding to each of the image sub-data, and sequentially arrange and combine each local key information description vector in the image key information description vector based on the sequence relationship of the corresponding image sub-data in the exemplary image data, so as to represent the image key information of each exemplary image in the exemplary image data.

2. The method of rail damage detection according to claim 1, wherein The exemplary image data includes a plurality of exemplary images, and the image segmentation operation of the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data includes: Determine a first image segmentation parameter and a second image segmentation parameter, wherein the first image segmentation parameter is used to represent the interval image frame number between two adjacent image sub-data formed by segmentation, and the second image segmentation parameter is used to represent the image frame number of the exemplary image included in the image sub-data formed by segmentation; Based on the first image segmentation parameter and the second image segmentation parameter, the image segmentation operation is performed on the exemplary image data to form a plurality of image sub-data corresponding to the exemplary image data.

3. The method of rail damage detection according to claim 1, wherein The image key information description vector includes a local key information description vector corresponding to each image sub-data; and the step of performing the merging operation on the user identity mapping vector to merge into the image key information description vector corresponding to the exemplary image data to form a corresponding multi-level merged description vector includes: Respectively perform the merging operation on the user identity mapping vector and the local key information description vector of each image sub-data to form a merged key information description vector corresponding to each image sub-data; Based on the sequence relationship of each image sub-data in the exemplary image data, sequentially arrange and combine the merged key information description vector corresponding to each image sub-data to form a multi-level merged description vector corresponding to the exemplary image data.

4. The method of rail damage detection according to claim 3, wherein, The local key information description vector corresponding to each image sub-data includes an image granularity level description vector of each exemplary image included in the corresponding image sub-data; and the to-be-processed image sub-data belongs to any one of the image sub-data included in the exemplary image data; The step of respectively performing the merging operation on the user identity mapping vector and the local key information description vector of each image sub-data to form a merged key information description vector corresponding to each image sub-data includes: Respectively perform the merging operation on the user identity mapping vector and the image granularity level description vector corresponding to each exemplary image included in the to-be-processed image sub-data to form a local multi-level merged description vector corresponding to each exemplary image, wherein the local multi-level merged description vector corresponding to the exemplary image simultaneously carries the image key information corresponding to the exemplary image and the user identity key information corresponding to the image analysis user; According to the local multi-level merged description vector corresponding to each exemplary image, combine to form a merged key information description vector corresponding to the to-be-processed image sub-data.

5. The method of rail damage detection according to claim 1, wherein, The step of performing network optimization operation on the initial image analysis network based on the image validity analysis data, the image key information description vector and the multi-level merged description vector to form a target image analysis network corresponding to the initial image analysis network comprises: performing validity analysis operation on the exemplary image data based on the image key information description vector and the multi-level merged description vector by the initial image analysis network to output image validity representation data corresponding to the exemplary image data; determining a network learning cost index corresponding to the initial image analysis network according to difference information between the image validity representation data and the image validity analysis data, and performing network optimization operation on the initial image analysis network based on the network learning cost index to form a target image analysis network corresponding to the initial image analysis network.

6. The method of rail flaw detection according to claim 5, wherein The step of performing validity analysis operation on the exemplary image data based on the image key information description vector and the multi-level merged description vector by the initial image analysis network to output image validity representation data corresponding to the exemplary image data comprises: performing validity analysis operation on the exemplary image data based on the image key information description vector by the initial image analysis network to output candidate validity representation data corresponding to the exemplary image data; performing validity analysis operation on the exemplary image data based on the multi-level merged description vector by the initial image analysis network to output pending validity representation data corresponding to the exemplary image data; analyzing the image validity representation data corresponding to the exemplary image data based on the candidate validity representation data and the pending validity representation data; The step of determining a network learning cost index corresponding to the initial image analysis network according to difference information between the image validity representation data and the image validity analysis data, and performing network optimization operation on the initial image analysis network based on the network learning cost index to form a target image analysis network corresponding to the initial image analysis network comprises: determining a network learning cost determination rule of the initial image analysis network; The image validity characterization data and the image validity analysis data are marked as to-be-processed data of the network learning cost determination rule, so as to determine the network learning cost index corresponding to the initial image analysis network, the network learning cost determination rule performs an analysis operation on a typical data set, the typical data set refers to a set of image validity analysis data of each exemplary image data by a plurality of image analysis users; the network learning cost determination rule includes a first network learning cost determination rule and a second network learning cost determination rule which are different from each other, when the distribution dispersion of the image validity analysis data included in the typical data set is greater than a preset distribution dispersion, the network learning cost index is calculated by the first network learning cost determination rule, and when the distribution dispersion of the image validity analysis data included in the typical data set is less than or equal to the preset distribution dispersion, the network learning cost index is calculated by the second network learning cost determination rule; Based on the network learning cost index, the initial image analysis network is subjected to network optimization operation to form a target image analysis network corresponding to the initial image analysis network.

7. The method of rail flaw detection according to claim 6, wherein The image key information description vector includes a local key information description vector corresponding to each image sub-data, and the step of performing validity analysis operation on the exemplary image data by the initial image analysis network according to the image key information description vector to output the candidate validity characterization data corresponding to the exemplary image data includes: The initial image analysis network is subjected to vector aggregation operation on the local key information description vector corresponding to each image sub-data to form a sub-data aggregation description vector corresponding to each image sub-data; According to the sub-data aggregation description vector corresponding to each image sub-data, the corresponding image sub-data is subjected to validity analysis operation to output the local validity characterization data corresponding to each image sub-data; The local validity characterization data corresponding to a plurality of image sub-data included in the exemplary image data is subjected to fusion operation to output the candidate validity characterization data corresponding to the exemplary image data.

8. The method of rail flaw detection according to any one of claims 1 to 7, wherein The step of performing damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output the target damage detection result corresponding to the to-be-processed rail includes: The target damage detection network formed by network optimization operation performs damage detection operation on the to-be-processed rail based on the target to-be-processed image data to output the target damage detection result corresponding to the to-be-processed rail, the target damage detection network is formed by performing network optimization operation on an initial damage detection network based on typical image data and actual damage state information of a typical rail corresponding to the typical image data; The rail damage detection method further includes: After obtaining the plurality of image data to be analyzed, each of the image data to be analyzed is analyzed by the target image analysis network to output target image validity analysis data corresponding to each of the image data to be analyzed, and the plurality of image data to be analyzed are formed by damage detection operation on the steel rail to be analyzed; Each of the image data to be analyzed is analyzed by the target damage detection network to output an initial damage detection result corresponding to each of the image data to be analyzed; Based on the target image validity analysis data corresponding to each of the analysis image data, the initial damage detection result corresponding to each of the image data to be analyzed is fused to output a target damage detection result corresponding to the steel rail to be analyzed.

9. A rail damage detection system characterised in that, The steel rail damage detection method comprises a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to realize the steel rail damage detection method in any one of claims 1-8.

Images