Performance data analysis method and system based on production line monitoring

Abstract

The application provides a performance data analysis method and system based on production line monitoring, and relates to the technical field of artificial intelligence.In the application, a production object performance analysis network is used to perform performance analysis operation on a production object performance image sequence, and output a first performance analysis result corresponding to a target production object; a performance image local sequence is screened from the production object performance image sequence, and a new production object performance image sequence is formed based on a determined associated performance image reference sequence of the performance image local sequence; the production object performance analysis network is used to perform performance analysis operation on the new production object performance image sequence, and output a second performance analysis result corresponding to the target production object; and the first performance analysis result and the second performance analysis result are fused to output a target performance analysis result.Based on the above, the reliability of production object performance analysis can be improved to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a performance data analysis method and system based on production line monitoring. Background Technology

[0002] By analyzing the performance data of the production object, performance analysis results can be obtained, such as the degree of performance excellence or specific performance descriptions. Specifically, continuous visual inspection can be performed on the production process and the state of the production object after production, such as image acquisition using image acquisition devices. The acquired images can then be analyzed to obtain corresponding performance analysis results. This analysis is generally for items with surface performance analysis, which can refer to surface defects, surface hardness, etc. However, in existing technologies, the reliability of performance analysis of production objects is generally low. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a performance data analysis method and system based on production line monitoring, so as to improve the reliability of performance analysis of production objects to a certain extent.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] A sequence of production object performance images is obtained by performing visual inspection on a target production object. A performance analysis network for the production object is then used to perform performance analysis on the sequence, outputting a first performance analysis result corresponding to the target production object. The production object performance image sequence includes multiple production object performance images, and each production object performance image serves as performance data for the target production object. The network optimization operation of the production object performance analysis network is based on an exemplary production object performance image sequence and the actual performance data of the exemplary production object performance image sequence. The network optimization operation includes: analyzing the performance analysis result of the exemplary production object performance image sequence using a candidate production object performance analysis network; and optimizing and adjusting the network parameters of the candidate production object performance analysis network based on the difference information between the performance analysis result and the actual performance data to reduce the difference information.

[0006] From the production object performance image sequence, select local performance image sequences, and based on the determined associated performance image reference sequences of the local performance image sequences, perform a replacement operation on the local performance image sequences in the production object performance image sequence to form a corresponding new production object performance image sequence.

[0007] Using the production object performance analysis network, a performance analysis operation is performed on the new production object performance image sequence, and a second performance analysis result corresponding to the target production object is output. Furthermore, a fusion operation is performed on the first performance analysis result and the second performance analysis result to output a target performance analysis result corresponding to the target production object. The target performance analysis result is used to reflect the surface performance of the target production object, and the target production object is a production object that requires surface performance analysis.

[0008] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the steps of filtering local performance image sequences from the performance image sequence of the production object, and replacing the local performance image sequences in the performance image sequence of the production object based on the determined associated performance image reference sequences of the local performance image sequences to form corresponding new performance image sequences of the production object, include:

[0009] The process involves filtering local performance image sequences from the production object performance image sequence, and mining key information representation vectors from the local performance image sequences. The local performance image sequences include multiple production object performance images, and the key information representation vectors from the local performance image sequences are used to reflect the image semantic information of the local performance image sequences.

[0010] Based on the key information representation vector of the local sequence of the performance image, the sequence identification data of the local sequence of the performance image is analyzed. The sequence identification data of the local sequence of the performance image includes the matching representation parameters of the local sequence of the performance image under multiple sequence type information. The matching representation parameters are used to reflect the degree of matching between the local sequence of the performance image and the sequence type information.

[0011] Based on the sequence identification data of the local performance image sequence and the sequence identification data of each performance image reference sequence included in the performance image reference sequence cluster, at least one undetermined performance image reference sequence is determined in the performance image reference sequence cluster.

[0012] In the at least one undetermined performance image reference sequence, an associated performance image reference sequence that has a correlation with the local performance image sequence is analyzed, and the local performance image sequence in the production object performance image sequence is replaced based on the associated performance image reference sequence to form a corresponding new production object performance image sequence.

[0013] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the step of analyzing the sequence identifier data of the local sequence of the performance image based on the key information representation vector of the local sequence of the performance image includes:

[0014] Using multiple different identifier data analysis models, based on the key information representation vector of the local sequence of the performance image, the identifier analysis data of the local sequence of the performance image is analyzed respectively. Different identifier data analysis models correspond to different analysis rules. The identifier analysis data analyzed by each identifier data analysis model includes the matching representation parameters of the local sequence of the performance image under the multiple sequence type information of the analysis rule corresponding to the identifier data analysis model.

[0015] Based on the identifier analysis data obtained from the various identifier data analysis models, the sequence identifier data of the local sequence of the performance image is output.

[0016] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the multiple different label data analysis models include a foreground label data analysis model and a background label data analysis model. The analysis rules corresponding to the foreground label data analysis model are for foreground image label analysis, and the analysis rules corresponding to the background label data analysis model are for background image label analysis.

[0017] The step of using multiple different identifier data analysis models to analyze the identifier analysis data of the local sequence of the performance image based on the key information representation vector of the local sequence of the performance image includes:

[0018] Using the foreground identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the foreground identifier analysis data of the local sequence of the performance image is analyzed. The foreground identifier analysis data includes the matching representation parameters of the local sequence of the performance image under the multiple sequence types of information formed by the foreground identifier analysis.

[0019] Using the background identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the background identifier analysis data of the local sequence of the performance image is analyzed. The background identifier analysis data includes the matching representation parameters of the local sequence of the performance image under the multiple sequence type information formed by the background image identifier analysis.

[0020] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the step of outputting sequence identification data of the local sequence of the performance image based on the identification analysis data analyzed by the multiple different identification data analysis models includes:

[0021] For each identifier data analysis model, based on the analysis data optimization rules corresponding to the identifier data analysis model, at least one matching characteristic parameter that matches the analysis data optimization rules is determined in the identifier analysis data analyzed by the identifier data analysis model, so as to form the optimized identifier analysis data corresponding to the identifier data analysis model;

[0022] The optimized identifier analysis data corresponding to each of the identifier data analysis models are merged to form the sequence identifier data of the local sequence of the performance image.

[0023] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the steps of filtering local performance image sequences from the performance image sequence of the production object and mining the key information representation vectors of the local performance image sequences include:

[0024] An image correlation analysis operation is performed on the multiple production object performance images included in the production object performance image sequence, and, based on the image correlation analysis results between adjacent production object performance images, a local performance image sequence is selected from the production object performance image sequence, wherein the multiple production object performance images included in the local performance image sequence have continuity in the production object performance image sequence.

[0025] The image multi-level information representation vector set of the local sequence of the performance image is mined. The image multi-level information representation vector set includes multiple image information representation vectors corresponding to each of the production object performance images formed by performing key information mining operations at the production object performance image granularity level on the local sequence of the performance image. The image information representation vectors are sorted in the image multi-level information representation vector set according to the sequence order of the corresponding production object performance images.

[0026] Based on the multi-level information representation vector set of the image, the key information representation vector of the local sequence of the performance image is determined.

[0027] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the step of mining the image multi-level information representation vector set of the local sequence of the performance image includes:

[0028] The temporal change information representation vector and the frequency change information representation vector corresponding to the local sequence of the performance image are extracted. The temporal change information representation vector is used to reflect the changes in image information of the local sequence of the performance image at the time level, and the frequency change information representation vector is used to reflect the changes in image information of the local sequence of the performance image at the frequency level.

[0029] At least one candidate time change information representation vector and at least one candidate frequency change information representation vector in the mining operation of the time change information representation vector are subjected to vector correlation analysis to form the correlation key information representation vector of the local sequence of the performance image. The correlation key information representation vector is used to reflect the correlation semantic information between the local sequence of the performance image at the time level and the frequency level.

[0030] Based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector, a set of multi-level information representation vectors for the local sequence of the performance image is determined.

[0031] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the at least one candidate time change information representation vector and the at least one candidate frequency change information representation vector form at least one information representation vector pair, and each information representation vector pair includes a pair of corresponding candidate time change information representation vectors and candidate frequency change information representation vectors.

[0032] The step of performing vector correlation analysis on at least one candidate time-varying information representation vector from the mining operation of the time-varying information representation vector and at least one candidate frequency-varying information representation vector from the mining operation of the frequency-varying information representation vector to form the associated key information representation vector of the local sequence of the performance image includes:

[0033] For each of the information representation vector pairs, a concatenated combination operation is performed on a pair of corresponding candidate time change information representation vectors and candidate frequency change information representation vectors included in the information representation vector pair to form the corresponding concatenated combination information representation vector of the information representation vector pair.

[0034] A key information mining operation is performed on each information representation vector in the at least one information representation vector pair to aggregate the corresponding cascaded combined information representation vectors, and the associated key information representation vector of the performance image local sequence is output.

[0035] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the step of mining the time variation information representation vector and frequency variation information representation vector corresponding to the local sequence of the performance image includes:

[0036] Using a key information mining model in the time dimension, the key information mining operation in the time dimension is performed on the local sequence of the performance image, and the time change information representation vector of the local sequence of the performance image is output.

[0037] The local sequence of the performance image is transformed by the image information dimension transformation operation, and the performance image frequency dimension information of the local sequence of the performance image is output.

[0038] Using a frequency-dimensional key information mining model, the frequency-dimensional information of the performance image is mined to output a frequency change information representation vector of the local sequence of the performance image.

[0039] Furthermore, the step of determining the multi-level information representation vector set of the performance image local sequence based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector includes:

[0040] The time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector are cascaded and combined to form a corresponding multi-dimensional key information representation vector.

[0041] Based on multiple pre-configured different information filtering rules, the multi-dimensional key information representation vector is subjected to information filtering operation to form multiple corresponding information filtering representation vectors.

[0042] Based on the multiple information filtering representation vectors, the image multi-level information representation vector set of the local sequence of the performance image is determined.

[0043] In some preferred embodiments, in the above-described performance data analysis method based on production line monitoring, the step of determining the key information representation vectors of the local sequence of the performance image based on the multi-level information representation vector set of the image includes:

[0044] Based on the first arrangement order, the multi-level information representation vector set of the image is subjected to association mining operation to form a corresponding first association mining feature representation. The first association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the first arrangement order.

[0045] Based on a second arrangement order that is opposite to the first arrangement order, the image multi-level information representation vector set is associated with the mining operation to form a corresponding second association mining feature representation. The second association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the second arrangement order.

[0046] The first association mining feature representation and the second association mining feature representation are cascaded and combined to form the key information representation vector of the local sequence of the performance image.

[0047] This invention also provides a performance data analysis system based on production line monitoring, including a processor and a memory. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described performance data analysis method based on production line monitoring.

[0048] The performance data analysis method and system based on production line monitoring provided in this invention first utilizes a production object performance analysis network to perform performance analysis on a production object performance image sequence, outputting a first performance analysis result corresponding to the target production object. Then, it filters local performance image sequences from the production object performance image sequence and, based on the determined associated performance image reference sequences, forms a new production object performance image sequence. Finally, it utilizes the production object performance analysis network to perform performance analysis on the new production object performance image sequence, outputting a second performance analysis result corresponding to the target production object. The first and second performance analysis results are then fused to output the target performance analysis result. Based on the foregoing, since a related associated performance image reference sequence is determined after the first performance analysis result is analyzed and replaced to form a new production object performance image sequence, a second performance analysis result can be further analyzed. The fusion of the two performance analysis results results in a more reliable target performance analysis result. Therefore, it can improve the reliability of production object performance analysis to a certain extent, addressing the problem of low reliability in existing technologies.

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0050] Figure 1 The structural block diagram of the performance data analysis system based on production line monitoring provided in the embodiments of the present invention is shown.

[0051] Figure 2 The flowchart illustrates the steps of the performance data analysis method based on production line monitoring provided in this embodiment of the invention.

[0052] Figure 3 This is a schematic diagram of the modules included in the performance data analysis device based on production line monitoring provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0055] like Figure 1 As shown in the figure, this embodiment of the invention provides a performance data analysis system based on production line monitoring. The performance data analysis system may include a memory and a processor.

[0056] In detail, the memory and the processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, they can be electrically connected via one or more communication buses or signal lines. The memory may store at least one software functional module (computer program) that exists in the form of software or firmware. The processor can be used to execute the executable computer program stored in the memory, thereby implementing the performance data analysis method based on production line monitoring provided in this embodiment of the invention.

[0057] It should be understood that, in one possible implementation, the memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor may be a general-purpose processor, including a Central Processing Unit (CPU), Network Processor (NP), System on Chip (SoC), etc.; it may also be a Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0058] It should be understood that, in one possible implementation, the performance data analysis system based on production line monitoring can be a server with data processing capabilities.

[0059] Combination Figure 2 This invention also provides a performance data analysis method based on production line monitoring, which can be applied to the aforementioned performance data analysis system based on production line monitoring. The method steps defined in the process of the performance data analysis method based on production line monitoring can be implemented by the performance data analysis system based on production line monitoring.

[0060] The following will be about Figure 2 The specific process shown will be explained in detail.

[0061] Step S100: Obtain the production object performance image sequence formed by performing visual inspection operation on the target production object; and use the production object performance analysis network formed by performing network optimization operation to perform performance analysis operation on the production object performance image sequence, and output the first performance analysis result corresponding to the target production object.

[0062] In this embodiment of the invention, the performance data analysis system based on production line monitoring can acquire a sequence of production object performance images formed by visual inspection of a target production object, and perform performance analysis on the production object performance image sequence using a production object performance analysis network formed through network optimization, outputting a first performance analysis result corresponding to the target production object. The production object performance image sequence includes multiple production object performance images (the multiple production object performance images can be sorted according to their formation order to form a corresponding image sequence), and each production object performance image serves as the performance data of the target production object. The network optimization operation of the production object performance analysis network is based on an exemplary production object performance image sequence and the actual performance data of the exemplary production object performance image sequence. The network optimization operation includes: analyzing the performance analysis result of the exemplary production object performance image sequence through a candidate production object performance analysis network, and optimizing and adjusting the network parameters of the candidate production object performance analysis network based on the difference information between the performance analysis result and the actual performance data to reduce the difference information.

[0063] Step S200: Select local performance image sequences from the production object performance image sequence, and based on the associated performance image reference sequence of the determined local performance image sequences, perform a replacement operation on the local performance image sequences in the production object performance image sequence to form a corresponding new production object performance image sequence.

[0064] In this embodiment of the invention, the performance data analysis system based on production line monitoring can filter local performance image sequences from the performance image sequence of the production object, and, based on the determined associated performance image reference sequence of the local performance image sequence (obtained by performing corresponding association analysis), perform a replacement operation on the local performance image sequence in the performance image sequence of the production object to form a corresponding new performance image sequence of the production object.

[0065] Step S300: Using the production object performance analysis network, perform performance analysis on the new production object performance image sequence, output the second performance analysis result corresponding to the target production object, and perform a fusion operation on the first performance analysis result and the second performance analysis result to output the target performance analysis result corresponding to the target production object.

[0066] In this embodiment of the invention, the performance data analysis system based on production line monitoring can utilize the production object performance analysis network to perform performance analysis on the new production object performance image sequence, output a second performance analysis result corresponding to the target production object, and perform a fusion operation on the first and second performance analysis results to output a target performance analysis result corresponding to the target production object. The target performance analysis result reflects the surface performance of the target production object; therefore, the target production object is a production object that requires surface performance analysis. For example, the target performance analysis result reflects the degree of performance excellence of the target production object. Thus, a weighted summation can be performed on the degree of performance excellence reflected by the first and second performance analysis results to obtain the degree of performance excellence reflected by the target performance analysis result. Exemplarily, during the weighted summation calculation, the weighting coefficient corresponding to the first performance analysis result can be greater than the weighting coefficient corresponding to the second performance analysis result.

[0067] For example, the surface properties of the target production object can be surface hardness, etc. (In other specific applications, it can also be other surface properties, which are not specifically limited here). That is to say, there is actually a correlation between the surface hardness of the production object and the specific information in the image. For example, for the surface of steel, a larger grain size leads to lower surface hardness, and a smaller grain size leads to higher surface hardness (similarly, for some non-metallic materials, such as plastics, the image information of their surface is also related to surface hardness). Based on this, the obtained performance image of the production object is formed by image acquisition using a high-resolution image acquisition device. Therefore, as long as the surface hardness can be reliably identified in the image during the neural network learning process (i.e., network optimization), allowing the neural network to learn the mapping relationship between different image information and surface hardness, the corresponding actual surface hardness can be analyzed in specific applications. It should be noted that the surface hardness analyzed in this way can serve as a reference. For example, after analyzing the surface hardness, it is also possible to determine whether other chemical or physical methods should be used for testing, or whether the production process of the production line should be analyzed, or whether the production process should be adjusted, etc., based on the specific value of the surface hardness.

[0068] Based on the foregoing content (i.e., steps S100-S300 above), after analyzing the first performance analysis result, a related associated performance image reference sequence will be determined for replacement operation, thereby forming a new production object performance image sequence. This allows for further analysis of the related second performance analysis result, and the two performance analysis results are then fused to obtain a more reliable target performance analysis result. Therefore, the reliability of production object performance analysis can be improved to a certain extent, thus addressing the problem of low reliability in the prior art.

[0069] It should be understood that, in one possible implementation, step S200 described above, namely, the step of filtering performance image local sequences from the production object performance image sequence and replacing the performance image local sequences in the production object performance image sequence based on the determined associated performance image reference sequences of the performance image local sequences to form a corresponding new production object performance image sequence, may further include the following steps: steps S120, S130, S140, and S150.

[0070] Step S120: Filter local performance image sequences from the performance image sequence of the production object, and mine the key information representation vector of the local performance image sequence.

[0071] In this embodiment of the invention, the performance data analysis system based on production line monitoring can filter local performance image sequences from the performance image sequence of the production object, and extract key information representation vectors from the local performance image sequences. The local performance image sequences include multiple performance images of the production object, and the key information representation vectors of the local performance image sequences are used to reflect the image semantic information of the local performance image sequences.

[0072] Step S130: Based on the key information representation vector of the local sequence of the performance image, analyze the sequence identification data of the local sequence of the performance image.

[0073] In this embodiment of the invention, the performance data analysis system based on production line monitoring can analyze the sequence identification data of the local performance image sequence based on the key information representation vector of the local performance image sequence. The sequence identification data of the local performance image sequence includes matching representation parameters corresponding to the local performance image sequence under multiple sequence type information, such as matching representation parameters under sequence type information 1, matching representation parameters under sequence type information 2, and matching representation parameters under sequence type information 3. The matching representation parameters are used to reflect the degree of matching between the local performance image sequence and the sequence type information.

[0074] Step S140: Based on the sequence identification data of the local performance image sequence and the sequence identification data of each performance image reference sequence included in the performance image reference sequence cluster, at least one undetermined performance image reference sequence is determined in the performance image reference sequence cluster.

[0075] In this embodiment of the invention, the performance data analysis system based on production line monitoring can determine at least one undetermined performance image reference sequence from the performance image reference sequence cluster based on the sequence identifier data of the local performance image sequence and the sequence identifier data of each performance image reference sequence included in the performance image reference sequence cluster. For example, for each performance image reference sequence, the similarity between the sequence identifier data of the performance image reference sequence and the sequence identifier data of the local performance image sequence (such as the average similarity of the matching characterization parameters corresponding to each sequence type information) can be calculated. Then, one or more performance image reference sequences with the highest similarity (the specific number can be configured according to actual needs) can be determined as undetermined performance image reference sequences. Alternatively, each performance image reference sequence with a similarity greater than a preset value can be used as an undetermined performance image reference sequence, such as 0.6, 0.7, etc.

[0076] Step S150: In the at least one undetermined performance image reference sequence, analyze the associated performance image reference sequence that has a correlation with the local performance image sequence, and perform a replacement operation on the local performance image sequence in the production object performance image sequence based on the associated performance image reference sequence to form a corresponding new production object performance image sequence.

[0077] In this embodiment of the invention, the performance data analysis system based on production line monitoring can analyze the at least one undetermined performance image reference sequence to identify associated performance image reference sequences that are related to the local performance image sequences. Furthermore, based on the associated performance image reference sequences, the system can replace the local performance image sequences in the production object performance image sequence to form a corresponding new production object performance image sequence, i.e., replacing the local performance image sequences with the associated performance image reference sequences. For example, for each undetermined performance image reference sequence, the key information representation vector of the undetermined performance image reference sequence can be first mined. Then, the vector distance (e.g., cosine distance) between the key information representation vector of the undetermined performance image reference sequence and the key information representation vector of the local performance image sequence can be calculated. Finally, the undetermined performance image reference sequence with the smallest vector distance can be considered as the associated performance image reference sequence that is related to the local performance image sequence.

[0078] It should be understood that, in one possible implementation, step S120 described above, namely, the step of filtering local performance image sequences from the production object performance image sequence and mining the key information representation vector of the local performance image sequence, may further include the following:

[0079] Image correlation analysis is performed on multiple production object performance images included in the production object performance image sequence. Based on the image correlation analysis results between adjacent production object performance images (such as the calculated image similarity), a local performance image sequence is selected from the production object performance image sequence. The multiple production object performance images included in the local performance image sequence are continuous in the production object performance image sequence. For example, the ratio of the number of production object performance images included in the local performance image sequence to the number of production object performance images included in the production object performance image sequence can be less than a preset ratio, such as 0.2. The average image similarity between any two adjacent production object performance images in the local performance image sequence can be greater than a preset similarity. The preset similarity can be configured according to actual needs, such as 0.5, 0.6, 0.7, etc.

[0080] A multi-level information representation vector set of the performance image local sequence is mined. The multi-level information representation vector set includes multiple image information representation vectors corresponding to each production object performance image, formed by performing key information mining operations at the production object performance image granularity level on the performance image local sequence. The image information representation vectors are ordered in the multi-level information representation vector set according to the sequential relationship of the corresponding production object performance images. For example, the image information representation vector corresponding to the first production object performance image is located in the first set position in the multi-level information representation vector set, the image information representation vector corresponding to the second production object performance image is located in the second set position, and the image information representation vector corresponding to the last production object performance image is located in the last set position.

[0081] Based on the multi-level information representation vector set of the image, the key information representation vector of the local sequence of the performance image is determined.

[0082] It should be understood that, in one possible implementation, the step of mining the image multi-level information representation vector set of the local sequence of the performance image may further include the following:

[0083] The time variation information representation vector and frequency variation information representation vector corresponding to the local sequence of the performance image are extracted. The time variation information representation vector is used to reflect the changes of image information in the local sequence of the performance image at the time level, that is, to describe the relationship between mathematical functions or physical signals and time. For example, the waveform of a signal can express the changes of the signal over time. The frequency variation information representation vector is used to reflect the changes of image information in the local sequence of the performance image at the frequency level, that is, to represent the relationship between frequency and amplitude.

[0084] At least one candidate time change information representation vector and at least one candidate frequency change information representation vector in the mining operation of the time change information representation vector are subjected to vector correlation analysis to form the correlation key information representation vector of the local sequence of the performance image. The correlation key information representation vector is used to reflect the correlation semantic information of the local sequence of the performance image at the time level and the frequency level, that is, it has image information at both the time level and the frequency level.

[0085] Based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector, a multi-level information representation vector set for the local sequence of the performance image is determined. For example, the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector can be fused.

[0086] It should be understood that, in one possible implementation, the step of mining the time-varying information representation vector and the frequency-varying information representation vector corresponding to the local sequence of the performance image may further include the following:

[0087] Using a key information mining model in the time dimension, the key information mining operation in the time dimension is performed on the local sequence of the performance image, and the time change information representation vector of the local sequence of the performance image is output.

[0088] The local sequence of the performance image is transformed by the image information dimension transformation operation, and the performance image frequency dimension information of the local sequence of the performance image is output. The performance image frequency dimension information can refer to the spectrum, that is, data describing the relationship between frequency and amplitude.

[0089] Using a frequency-dimensional key information mining model, the frequency-dimensional information of the performance image is mined to output a frequency change information representation vector of the local sequence of the performance image.

[0090] It should be understood that, in one possible implementation, the step of using a time-dimensional key information mining model to perform time-dimensional key information mining on the local sequence of the performance image and outputting a time-varying information representation vector of the local sequence of the performance image may further include the following:

[0091] The first convolutional unit (which may include one or more convolutional kernels) in the key information mining model of the time dimension is used to perform convolution operation on the local sequence of the performance image to output the convolution result representation vector corresponding to the convolutional unit. The information compression operation (such as POOLing) is performed on the convolution result representation vector to output the compressed representation vector corresponding to the convolutional unit.

[0092] For each convolutional unit other than the first convolutional unit in the key information mining model of the time dimension, the convolutional unit is used to perform a convolution operation on the compressed representation vector corresponding to the previous convolutional unit, and the convolution result representation vector corresponding to the convolutional unit is output. In addition, the information compression operation is performed on the convolution result representation vector, and the compressed representation vector corresponding to the convolutional unit is output.

[0093] After obtaining the compressed representation vector corresponding to the last convolutional unit in the key information mining model of the time dimension, the temporal change information representation vector of the local sequence of the performance image is determined based on the compressed representation vector. For example, the compressed representation vector can be used as the temporal change information representation vector of the local sequence of the performance image. The key information mining model of the time dimension includes multiple convolutional units, which are connected sequentially.

[0094] It should be understood that, in one possible implementation, the step of using a frequency-dimensional key information mining model to perform frequency-dimensional key information mining operations on the performance image frequency-dimensional information and outputting the frequency change information representation vector of the local sequence of the performance image may further include the following:

[0095] The first convolutional unit (which may include one or more convolutional kernels) in the key information mining model of the frequency dimension is used to perform convolution operation on the frequency dimension information of the performance image to output the convolution result representation vector corresponding to the convolutional unit. In addition, information compression operation (such as POOLing) is performed on the convolution result representation vector to output the compressed representation vector corresponding to the convolutional unit.

[0096] For each convolutional unit other than the first convolutional unit in the key information mining model of the frequency dimension, the convolutional unit is used to perform a convolution operation on the compressed representation vector corresponding to the previous convolutional unit, and the convolution result representation vector corresponding to the convolutional unit is output. In addition, the information compression operation is performed on the convolution result representation vector, and the compressed representation vector corresponding to the convolutional unit is output.

[0097] After obtaining the compressed representation vector corresponding to the last convolutional unit in the frequency dimension key information mining model, the frequency change information representation vector of the local sequence of the performance image is determined based on the compressed representation vector. For example, the compressed representation vector can be used as the frequency change information representation vector of the local sequence of the performance image. The frequency dimension key information mining model includes multiple convolutional units, which are connected sequentially.

[0098] It should be understood that, in one possible implementation, the step of determining the multi-level information representation vector set of the performance image local sequence based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector may further include the following:

[0099] The time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector are concatenated and combined to form a corresponding multi-dimensional key information representation vector. That is, the multi-dimensional key information representation vector can be {the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector}. In some embodiments, the vector dimensions of the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector can be adjusted. In this way, the vector dimensions of the adjusted time-varying information representation vector, the adjusted frequency-varying information representation vector, and the adjusted associated key information representation vector can be made consistent, which facilitates the concatenation and combination of vectors.

[0100] Based on multiple pre-configured information filtering rules, the multidimensional key information representation vectors are subjected to information filtering operations to form multiple corresponding information filtering representation vectors. The information filtering operation can refer to the aforementioned information compression operation. For example, in the first information filtering rule, the maximum value of each vector parameter within the sliding window (i.e., based on the sliding window operation on the multidimensional key information representation vectors) can be used as the representative vector parameter of the sliding window. Thus, the representative vector parameters can be combined to form an information filtering representation vector. In the second information filtering rule, the average value of each vector parameter within the sliding window can be used as the representative vector parameter of the sliding window. Thus, the representative vector parameters can be combined to form an information filtering representation vector. In the third information filtering rule, the median value of each vector parameter within the sliding window can be used as the representative vector parameter of the sliding window. Thus, the representative vector parameters can be combined to form an information filtering representation vector. In other embodiments, the information compression operation may have different logic than described above.

[0101] Based on the multiple information filtering representation vectors, a multi-level information representation vector set for the local sequence of performance images is determined. For example, the above operation can be performed on each production object performance image. In this way, the multiple information filtering representation vectors corresponding to each production object performance image can be obtained. For each production object performance image, the multiple information filtering representation vectors corresponding to the production object performance image can be superimposed first. Then, a fully connected operation can be performed on the result of the superposition operation. Finally, the results of the fully connected operation corresponding to each production object performance image can be arranged in order to form the multi-level information representation vector set for the local sequence of performance images. The result of the fully connected operation can be used as the image information representation vector corresponding to the production object performance image.

[0102] It should be understood that, in one possible implementation, the at least one candidate temporal change information representation vector (such as the compressed representation vectors corresponding to each convolutional unit other than the last one included in the aforementioned key information mining model of the time dimension) and the at least one candidate frequency change information representation vector (such as the compressed representation vectors corresponding to each convolutional unit other than the last one included in the aforementioned key information mining model of the frequency dimension) form at least one information representation vector pair. Each information representation vector pair includes a corresponding pair of candidate temporal change information representation vectors and candidate frequency change information representation vectors (such as the compressed representation vectors corresponding to the first convolutional unit included in the aforementioned key information mining model of the time dimension and the compressed representation vectors corresponding to the first convolutional unit included in the aforementioned key information mining model of the frequency dimension). The compressed representation vectors corresponding to the first convolutional unit of the mining model can form an information representation vector pair; the compressed representation vectors corresponding to the second convolutional unit of the aforementioned time-dimensional key information mining model and the compressed representation vectors corresponding to the second convolutional unit of the aforementioned frequency-dimensional key information mining model can form an information representation vector pair. Based on this, the step of performing vector correlation analysis on at least one candidate time-varying information representation vector in the mining operation of the time-varying information representation vectors and at least one candidate frequency-varying information representation vector in the mining operation of the frequency-varying information representation vectors to form the associated key information representation vector of the local sequence of the performance image can further include the following:

[0103] For each of the information representation vector pairs, a concatenated combination operation is performed on a pair of corresponding candidate time change information representation vectors and candidate frequency change information representation vectors included in the information representation vector pair (or the vector dimensions can be adjusted first) to form the corresponding concatenated combination information representation vectors of the information representation vector pair. For example, concatenated combination information representation vector 1 can be {candidate time change information representation vector 1, candidate frequency change information representation vector 1}, concatenated combination information representation vector 2 can be {candidate time change information representation vector 2, candidate frequency change information representation vector 2}, and concatenated combination information representation vector 3 can be {candidate time change information representation vector 3, candidate frequency change information representation vector 3}.

[0104] A key information mining operation is performed on each information representation vector in the at least one information representation vector pair to aggregate the corresponding cascaded combined information representation vectors, and the associated key information representation vector of the performance image local sequence is output.

[0105] It should be understood that, in one possible implementation, the step of performing a key information mining operation on each of the at least one information representation vector pair to aggregate the corresponding cascaded combined information representation vectors, and outputting the associated key information representation vector of the local sequence of the performance image, may further include the following:

[0106] Determine the order of forming each of the at least one information representation vector pairs, and, for the second formed information representation vector pair, perform an aggregation operation on the corresponding concatenated information representation vector of the information representation vector pair and the corresponding concatenated information representation vector of the previous information representation vector pair, such as performing a concatenated combination operation, and then perform a convolution operation on the result of the aggregation operation to output the convolution vector corresponding to the current information representation vector pair.

[0107] For each subsequent information representation vector pair formed after the second information representation vector pair, the corresponding concatenated combination information representation vector of the information representation vector pair and the corresponding convolution vector of the previous information representation vector pair are aggregated, such as by performing a concatenated combination operation. Then, the result of the aggregation operation is convolved to output the corresponding convolution vector of the current information representation vector pair.

[0108] After determining the last formed information representation vector pair and its corresponding convolution vector, the last formed information representation vector pair and its corresponding convolution vector are labeled as the associated key information representation vector of the local sequence of the performance image.

[0109] It should be understood that, in one possible implementation, the step of determining the key information representation vectors of the local sequence of the performance image based on the multi-level information representation vector set of the image may further include the following:

[0110] Based on the first arrangement order, the multi-level information representation vector set of the image is subjected to association mining operation to form a corresponding first association mining feature representation. The first association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the first arrangement order.

[0111] Based on the second arrangement order relationship which is opposite to the first arrangement order relationship, the image multi-level information representation vector set is associated with the mining operation to form a corresponding second association mining feature representation. The second association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the second arrangement order relationship. The specific way of forming the first association mining feature representation by the association mining operation in the following text can be referred to.

[0112] The first association mining feature representation and the second association mining feature representation are cascaded and combined to form the key information representation vector of the local sequence of the performance image. That is, the key information representation vector of the local sequence of the performance image can be {the first association mining feature representation and the second association mining feature representation}.

[0113] It should be understood that, in one possible implementation, as mentioned above, the result of the fully connected operation can be used as the image information representation vector corresponding to the performance image of the production object. That is, the image multi-level information representation vector set includes multiple image information representation vectors arranged sequentially. Based on this, the step of performing association mining operation on the image multi-level information representation vector set based on the first arrangement order to form the corresponding first association mining feature representation may further include the following:

[0114] For the first image information representation vector in the multi-level image information representation vector set, based on the image information representation vector, a focusing feature analysis operation is performed on the image information representation vector to form a focusing feature representation vector corresponding to the image information representation vector. In addition, the concatenated combination result of the focusing feature representation vector and the image information representation vector is convolved to form the convolution result vector corresponding to the image information representation vector.

[0115] For each image information representation vector other than the first image information representation vector in the image multi-level information representation vector set, based on the convolution result vector corresponding to the previous image information representation vector of the other image information representation vector, a focusing feature analysis operation is performed on the other image information representation vector to form a focusing feature representation vector corresponding to the other image information representation vector. In addition, a convolution operation is performed on the concatenated combination result of the focusing feature representation vector and the other image information representation vector to form a convolution result vector corresponding to the other image information representation vector.

[0116] After obtaining the convolution result vector corresponding to the last image information representation vector, the convolution result vector corresponding to the last image information representation vector is used as the corresponding first association mining feature representation.

[0117] It should be understood that, in one possible implementation, step S130 described above, namely the step of analyzing the sequence identifier data of the local sequence of the performance image based on the key information representation vector of the local sequence of the performance image, may further include the following:

[0118] Using multiple different identifier data analysis models, based on the key information representation vector of the local sequence of the performance image, the identifier analysis data of the local sequence of the performance image is analyzed respectively. Different identifier data analysis models correspond to different analysis rules. The identifier analysis data analyzed by each identifier data analysis model includes the matching representation parameters of the local sequence of the performance image under the multiple sequence type information of the analysis rule corresponding to the identifier data analysis model. In the process of network optimization, each identifier data analysis model can learn based on the key information representation vector of the exemplary local sequence of the performance image and the corresponding actual identifier data, so that each identifier data analysis model can learn different mapping relationships.

[0119] Based on the identifier analysis data obtained from the various identifier data analysis models, the sequence identifier data of the local sequence of the performance image is output. For example, the identifier analysis data obtained from the various identifier data analysis models can be directly merged to obtain the sequence identifier data of the local sequence of the performance image.

[0120] It should be understood that, in one possible implementation, the multiple different identifier data analysis models include a foreground identifier data analysis model and a background identifier data analysis model. The analysis rules corresponding to the foreground identifier data analysis model are for foreground image identifier analysis, and the analysis rules corresponding to the background identifier data analysis model are for background image identifier analysis. Based on this, the step of using multiple different identifier data analysis models to analyze the identifier analysis data of the local sequence of the performance image based on the key information representation vector of the local sequence of the performance image may further include the following:

[0121] Using the foreground identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the foreground identifier analysis data of the local sequence of the performance image is analyzed. The foreground identifier analysis data includes the matching representation parameters corresponding to each of the multiple sequence type information formed by the foreground image identifier analysis of the local sequence of the performance image. For example, the foreground image can refer to the image area where the production object is located, and the corresponding sequence type information can refer to information such as the shape of the production object.

[0122] Using the background identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the background identifier analysis data of the local sequence of the performance image is analyzed. The background identifier analysis data includes the matching representation parameters corresponding to each of the multiple sequence type information formed by the background image identifier analysis of the local sequence of the performance image. For example, the background image can refer to other image areas outside the image area where the production object is located, such as the image information of the production equipment of the production object. The corresponding sequence type information can refer to information such as the shape of the equipment.

[0123] It should be understood that, in one possible implementation, the step of outputting the sequence identification data of the local sequence of the performance image based on the identification analysis data analyzed by the multiple different identification data analysis models may further include the following:

[0124] For each identifier data analysis model, based on the analysis data optimization rules corresponding to the identifier data analysis model, at least one matching characteristic parameter that matches the analysis data optimization rules is determined in the identifier analysis data analyzed by the identifier data analysis model, so as to form the optimized identifier analysis data corresponding to the identifier data analysis model. For example, the largest one or a specified number of matching characteristic parameters can be selected, and the specific number can be configured according to the requirements.

[0125] The optimized identifier analysis data corresponding to each of the identifier data analysis models are merged to form the sequence identifier data of the local sequence of the performance image. In other words, the sequence identifier data of the local sequence of the performance image includes the optimized identifier analysis data corresponding to each of the identifier data analysis models. This can reduce the interference of invalid data.

[0126] Combination Figure 3 This invention also provides a performance data analysis device based on production line monitoring, which can be applied to the aforementioned performance data analysis system based on production line monitoring. The performance data analysis device based on production line monitoring may include the following software functional modules:

[0127] An initial performance analysis module is used to acquire a sequence of production object performance images formed by visual inspection of a target production object, and to perform performance analysis on the production object performance image sequence using a production object performance analysis network formed by network optimization, and output a first performance analysis result corresponding to the target production object. The production object performance image sequence includes multiple production object performance images, and each production object performance image serves as the performance data of the target production object. The network optimization operation of the production object performance analysis network is based on an exemplary production object performance image sequence and the actual performance data of the exemplary production object performance image sequence. The network optimization operation includes: analyzing the performance analysis result of the exemplary production object performance image sequence through a candidate production object performance analysis network, and optimizing and adjusting the network parameters of the candidate production object performance analysis network based on the difference information between the performance analysis result and the actual performance data to reduce the difference information.

[0128] The performance image sequence update module is used to filter local performance image sequences from the production object performance image sequence, and, based on the determined associated performance image reference sequence of the local performance image sequence, perform a replacement operation on the local performance image sequence in the production object performance image sequence to form a corresponding new production object performance image sequence.

[0129] The target performance analysis module is used to perform performance analysis on the new production object performance image sequence using the production object performance analysis network, output a second performance analysis result corresponding to the target production object, and perform a fusion operation on the first performance analysis result and the second performance analysis result to output a target performance analysis result corresponding to the target production object. The target performance analysis result is used to reflect the surface performance of the target production object.

[0130] In summary, the performance data analysis method and system based on production line monitoring provided by this invention can first perform performance analysis on the performance image sequence of the production object and output a first performance analysis result; then, select local performance image sequences from the performance image sequence of the production object and analyze the corresponding sequence identification data; determine a reference sequence of performance images to be determined based on the sequence identification data; analyze related performance image reference sequences from the reference sequence of performance images to be determined, and form a new performance image sequence of the production object based on the related performance image reference sequences; perform performance analysis on the new performance image sequence of the production object and output a second performance analysis result; and then fuse the first and second performance analysis results to output a target performance analysis result. Based on the foregoing, since related related performance image reference sequences are determined after the first performance analysis result is analyzed and replaced to form a new performance image sequence of the production object, a second performance analysis result can be further analyzed. The fusion of the two performance analysis results results in a more reliable target performance analysis result. Therefore, it can improve the reliability of production object performance analysis to a certain extent, thereby addressing the problem of low reliability in existing production object performance analysis technologies.

[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A performance data analysis method based on production line monitoring, characterized in that, include: A sequence of production object performance images is obtained by performing visual inspection on a target production object. A performance analysis operation is performed on the production object performance image sequence using a production object performance analysis network formed through network optimization. A first performance analysis result corresponding to the target production object is output. The production object performance image sequence includes multiple production object performance images. The network optimization operation of the production object performance analysis network is based on an exemplary production object performance image sequence and the actual performance data of the exemplary production object performance image sequence. The network optimization operation includes: analyzing the performance analysis result of the exemplary production object performance image sequence using a candidate production object performance analysis network; and optimizing and adjusting the network parameters of the candidate production object performance analysis network based on the difference information between the performance analysis result and the actual performance data to reduce the difference information. From the production object performance image sequence, select local performance image sequences, and based on the determined associated performance image reference sequences of the local performance image sequences, perform a replacement operation on the local performance image sequences in the production object performance image sequence to form a corresponding new production object performance image sequence. Using the production object performance analysis network, a performance analysis operation is performed on the new production object performance image sequence, and a second performance analysis result corresponding to the target production object is output. Furthermore, a fusion operation is performed on the first performance analysis result and the second performance analysis result to output a target performance analysis result corresponding to the target production object. The target performance analysis result is used to reflect the surface performance of the target production object, and the target production object is a production object that requires surface performance analysis.

2. The performance data analysis method based on production line monitoring as described in claim 1, characterized in that, The steps of filtering local performance image sequences from the production object performance image sequence and replacing the local performance image sequences in the production object performance image sequence based on the determined associated performance image reference sequences to form corresponding new production object performance image sequences include: The process involves filtering local performance image sequences from the production object performance image sequence, and mining key information representation vectors from the local performance image sequences. The local performance image sequences include multiple production object performance images, and the key information representation vectors from the local performance image sequences are used to reflect the image semantic information of the local performance image sequences. Based on the key information representation vector of the local sequence of the performance image, the sequence identification data of the local sequence of the performance image is analyzed. The sequence identification data of the local sequence of the performance image includes the matching representation parameters of the local sequence of the performance image under multiple sequence type information. The matching representation parameters are used to reflect the degree of matching between the local sequence of the performance image and the sequence type information. Based on the sequence identification data of the local performance image sequence and the sequence identification data of each performance image reference sequence included in the performance image reference sequence cluster, at least one undetermined performance image reference sequence is determined in the performance image reference sequence cluster. In the at least one undetermined performance image reference sequence, an associated performance image reference sequence that has a correlation with the local performance image sequence is analyzed, and the local performance image sequence in the production object performance image sequence is replaced based on the associated performance image reference sequence to form a corresponding new production object performance image sequence.

3. The performance data analysis method based on production line monitoring as described in claim 2, characterized in that, The step of analyzing the sequence identifier data of the local sequence of the performance image based on the key information representation vector of the local sequence includes: Using multiple different identifier data analysis models, based on the key information representation vector of the local sequence of the performance image, the identifier analysis data of the local sequence of the performance image is analyzed respectively. Different identifier data analysis models correspond to different analysis rules. The identifier analysis data analyzed by each identifier data analysis model includes the matching representation parameters of the local sequence of the performance image under the multiple sequence type information of the analysis rule corresponding to the identifier data analysis model. Based on the identifier analysis data obtained from the various identifier data analysis models, the sequence identifier data of the local sequence of the performance image is output.

4. The performance data analysis method based on production line monitoring as described in claim 3, characterized in that, The multiple different identifier data analysis models include a foreground identifier data analysis model and a background identifier data analysis model. The analysis rules corresponding to the foreground identifier data analysis model are for foreground image identifier analysis, and the analysis rules corresponding to the background identifier data analysis model are for background image identifier analysis. The step of using multiple different identifier data analysis models to analyze the identifier analysis data of the local sequence of the performance image based on the key information representation vector of the local sequence of the performance image includes: Using the foreground identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the foreground identifier analysis data of the local sequence of the performance image is analyzed. The foreground identifier analysis data includes the matching representation parameters of the local sequence of the performance image under the multiple sequence types of information formed by the foreground identifier analysis. Using the background identifier data analysis model, based on the key information representation vector of the local sequence of the performance image, the background identifier analysis data of the local sequence of the performance image is analyzed. The background identifier analysis data includes the matching representation parameters of the local sequence of the performance image under the multiple sequence type information formed by the background image identifier analysis. Furthermore, the step of outputting the sequence identification data of the local sequence of the performance image based on the identification analysis data analyzed by the multiple different identification data analysis models includes: For each identifier data analysis model, based on the analysis data optimization rules corresponding to the identifier data analysis model, at least one matching characteristic parameter that matches the analysis data optimization rules is determined in the identifier analysis data analyzed by the identifier data analysis model, so as to form the optimized identifier analysis data corresponding to the identifier data analysis model; The optimized identifier analysis data corresponding to each of the identifier data analysis models are merged to form the sequence identifier data of the local sequence of the performance image.

5. The performance data analysis method based on production line monitoring as described in claim 2, characterized in that, The steps of filtering local performance image sequences from the performance image sequence of the production object and mining the key information representation vectors of the local performance image sequences include: An image correlation analysis operation is performed on the multiple production object performance images included in the production object performance image sequence, and, based on the image correlation analysis results between adjacent production object performance images, a local performance image sequence is selected from the production object performance image sequence, wherein the multiple production object performance images included in the local performance image sequence have continuity in the production object performance image sequence. The image multi-level information representation vector set of the local sequence of the performance image is mined. The image multi-level information representation vector set includes multiple image information representation vectors corresponding to each of the production object performance images formed by performing key information mining operations at the production object performance image granularity level on the local sequence of the performance image. The image information representation vectors are sorted in the image multi-level information representation vector set according to the sequence order of the corresponding production object performance images. Based on the multi-level information representation vector set of the image, the key information representation vector of the local sequence of the performance image is determined.

6. The performance data analysis method based on production line monitoring as described in claim 5, characterized in that, The step of mining the image multi-level information representation vector set of the local sequence of the performance image includes: The temporal change information representation vector and the frequency change information representation vector corresponding to the local sequence of the performance image are extracted. The temporal change information representation vector is used to reflect the changes in image information of the local sequence of the performance image at the time level, and the frequency change information representation vector is used to reflect the changes in image information of the local sequence of the performance image at the frequency level. At least one candidate time change information representation vector and at least one candidate frequency change information representation vector in the mining operation of the time change information representation vector are subjected to vector correlation analysis to form the correlation key information representation vector of the local sequence of the performance image. The correlation key information representation vector is used to reflect the correlation semantic information between the local sequence of the performance image at the time level and the frequency level. Based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector, a set of multi-level information representation vectors for the local sequence of the performance image is determined.

7. The performance data analysis method based on production line monitoring as described in claim 6, characterized in that, The at least one candidate time change information representation vector and the at least one candidate frequency change information representation vector form at least one information representation vector pair, and each information representation vector pair includes a corresponding pair of candidate time change information representation vectors and candidate frequency change information representation vectors; The step of performing vector correlation analysis on at least one candidate time-varying information representation vector from the mining operation of the time-varying information representation vector and at least one candidate frequency-varying information representation vector from the mining operation of the frequency-varying information representation vector to form the associated key information representation vector of the local sequence of the performance image includes: For each of the information representation vector pairs, a concatenated combination operation is performed on a pair of corresponding candidate time change information representation vectors and candidate frequency change information representation vectors included in the information representation vector pair to form the corresponding concatenated combination information representation vector of the information representation vector pair. A key information mining operation is performed on each information representation vector in the at least one information representation vector pair to aggregate the corresponding cascaded combined information representation vectors, and the associated key information representation vector of the performance image local sequence is output.

8. The performance data analysis method based on production line monitoring as described in claim 6, characterized in that, The step of mining the time-varying information representation vector and frequency-varying information representation vector corresponding to the local sequence of the performance image includes: Using a key information mining model in the time dimension, the key information mining operation in the time dimension is performed on the local sequence of the performance image, and the time change information representation vector of the local sequence of the performance image is output. The local sequence of the performance image is transformed by the image information dimension transformation operation, and the performance image frequency dimension information of the local sequence of the performance image is output. Using a frequency-dimensional key information mining model, the frequency-dimensional information of the performance image is mined to output a frequency change information representation vector of the local sequence of the performance image. Furthermore, the step of determining the multi-level information representation vector set of the performance image local sequence based on the time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector includes: The time-varying information representation vector, the frequency-varying information representation vector, and the associated key information representation vector are cascaded and combined to form a corresponding multi-dimensional key information representation vector. Based on multiple pre-configured different information filtering rules, the multi-dimensional key information representation vector is subjected to information filtering operation to form multiple corresponding information filtering representation vectors. Based on the multiple information filtering representation vectors, the image multi-level information representation vector set of the local sequence of the performance image is determined.

9. The performance data analysis method based on production line monitoring as described in claim 5, characterized in that, The step of determining the key information representation vectors of the local sequence of the performance image based on the multi-level information representation vector set of the image includes: Based on the first arrangement order, the multi-level information representation vector set of the image is subjected to association mining operation to form a corresponding first association mining feature representation. The first association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the first arrangement order. Based on a second arrangement order that is opposite to the first arrangement order, the image multi-level information representation vector set is associated with the mining operation to form a corresponding second association mining feature representation. The second association mining feature representation is used to reflect the image semantic information of the local sequence of the performance image in the second arrangement order. The first association mining feature representation and the second association mining feature representation are cascaded and combined to form the key information representation vector of the local sequence of the performance image.

10. A performance data analysis system based on production line monitoring, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to implement the method of any one of claims 1-9.

Images