Method, apparatus, electronic device and storage medium for detecting spinning workplace

Abstract

A method, apparatus, electronic device and storage medium for detecting a spinning workplace are provided. The present disclosure provides an automated monitoring system for a spinning workshop, comprising: performing image collection for a process control device of a spinning box in a spinning workshop to obtain an image to be processed; extracting a first image feature from the image to be processed; and processing the first image feature based on a decoder network to obtain a fault detection result, the decoder network includes a plurality of decoder modules connected in series in sequence, and for each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head classifies and predicts the output feature of the decoder layer to obtain a first fault classification result, and the fault detection result output by the decoder network includes a second fault classification result output by a decoder module of the last layer of the decoder network.

Description

[Technical field]

[0001] The present disclosure relates to the field of data processing, and in particular to the fields of artificial intelligence, computer vision, and the like. [Background technology]

[0002] In the industrial scenario of the spinning process, the spinning box in the spinning workshop plays a vital role in the manufacturing process of chemical fiber products. However, the efficiency of manual spinning box inspection is low, so how to efficiently inspect the spinning workshop is a problem currently faced. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a spinning workplace detection method, apparatus, electronic device and storage medium to solve or alleviate one or more technical problems in the prior art. [Means for solving the problem]

[0004] In a first aspect, the present disclosure provides a method for detecting a spinning workplace, the method comprising: performing image acquisition on a process control device of a spinning box in a spinning workshop to obtain images to be processed; Extracting a first image feature from the image to be processed; processing the first image feature based on a decoder network to obtain a fault detection result for the process control device; The decoder network includes a plurality of decoder modules connected in series, For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result, and update the first fault classification result according to a preset classification calibration matrix to obtain a second fault classification result output by the decoder module; The fault detection result output by the decoder network includes a second fault classification result output by the decoder module in the last layer of the decoder network and a fault location output by the decoder module in the last layer.

[0005] In a second aspect, the present disclosure provides a detection device for a spinning workshop, the device comprising: a collection unit for performing image collection for a process control device of a spinning box in a spinning workshop to obtain images to be processed; an extraction unit for extracting a first image feature from the image to be processed; a processing unit for processing the first image feature based on the decoder network to obtain a fault detection result for the process control device; The decoder network includes a plurality of decoder modules connected in series, For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result, and update the first fault classification result according to a preset classification calibration matrix to obtain a second fault classification result output by the decoder module; The fault detection result output by the decoder network includes a second fault classification result output by the decoder module in the last layer of the decoder network and a fault location output by the decoder module in the last layer.

[0006] In a third aspect, the present disclosure provides an electronic device, the device comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0007] In a fourth aspect, there is provided a non-transitory computer readable storage medium having computer instructions stored thereon for causing a computer to perform any one of the methods according to the embodiments of the present disclosure.

[0008] In a fifth aspect, there is provided a program product comprising a program for performing any one of the methods of the embodiments of the present disclosure when executed by a processor.

[0009] The embodiments of the present disclosure realize automated monitoring of spinning boxes in spinning workshops, which is helpful in finding fault areas and fault types in a timely manner.

[0010] It should be understood that the descriptions herein are not intended to describe key or important features of the embodiments of the present disclosure, nor are they intended to be used to limit the scope of the present disclosure. Other features of the present disclosure are best understood through the following specification. [Brief description of the drawings]

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the several accompanying drawings indicate the same or similar components or elements. The accompanying drawings are not necessarily drawn to scale. It should be understood that the drawings illustrate only some examples provided by the present disclosure and should not be considered as limiting the scope of the present disclosure.

[0012] [Figure 1] 1 is a flow chart of a method for detecting a spinning workplace according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram of a decoder network according to one embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of obtaining a mask image according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is another schematic diagram of a decoder network according to an embodiment of the present disclosure. [Diagram 5]FIG. 2 is a schematic diagram of the overall framework of a spinning workplace detection method according to an embodiment of the present disclosure; [Figure 6] FIG. 2 is a block diagram of a detection device in a spinning workshop according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of an electronic device for implementing a spinning workplace detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present disclosure will now be described in more detail with reference to the accompanying drawings, in which like reference numbers represent functionally identical or similar elements, and in which various aspects of illustrative embodiments are illustrated, and which, unless otherwise noted, are not necessarily drawn to scale.

[0014] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can be similarly implemented without specific details. In some embodiments, methods, means, components, circuits, etc. that are well known to those skilled in the art are not described in detail so as not to obscure the gist of the present disclosure.

[0015] Additionally, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance, nor do they implicitly designate the number of technical features depicted. Thus, a feature defined with the term "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "one or more" means two or more, unless expressly and specifically limited otherwise.

[0016] In the industrial scenario of the spinning process, the spinning box in the spinning workshop plays a vital role in the manufacturing process of chemical fiber products. The important components of the spinning box include the spinning component and the process control device on the back of the spinning component. The important parts of the process control device in the embodiment of the present disclosure include the heating device, the melt distribution pipe, the auxiliary material transport pipe, the metering pump, etc., where the melt distribution pipe has a heat-insulating layer, and the heating device needs to ensure the temperature of the melt in the melt distribution pipe to facilitate the distribution of the melt to the metering pump. The metering pump distributes the melt evenly to each component of the spinning box, and forms a melt trickle through the pores of the spinneret in the spinning component, and the melt trickle is formed into a yarn by natural cooling after leaving the spinneret.

[0017] Here, the melt needs to be kept warm in the piping of the spinning box in order to ensure as uniform a temperature as possible.

[0018] The embodiments of the present disclosure are mainly related to fault detection for the process control devices behind the spinning components, especially for the heaters, melt distribution pipes, and auxiliary material transport pipes. For ease of explanation and understanding, the embodiments of the present disclosure collectively refer to the heaters and melt distribution pipes as a temperature control system.

[0019] In addition, the embodiments of the present disclosure can also realize the detection of open areas on the back side of the spinning components, etc., to improve the detection efficiency of the spinning workplace.

[0020] In addition, the main types of spun products according to the solutions of the embodiments of the present disclosure may include one or more of pre-oriented yarns (Partially Oriented Yarns, POY), fully draft yarns (Fully Drawn Yarns, FDY), polyester staple fibers, etc. For example, specific types of yarns may include polyester partially oriented yarns, polyester fully draft yarns, polyester draft yarns, polyester staple fibers, etc.

[0021] In order to automatically and accurately detect the spinning workshop, in the embodiment of the present disclosure, a method for detecting the spinning workshop is proposed using DETR (DEtection Transformer, target detection architecture), as shown in FIG. 1, which includes:

[0022] In S101, image collection is performed for a process control device of a spinning box in a spinning workshop to obtain an image to be processed.

[0023] Here, the drone can be used to collect images of the process control device of the spinning box in the spinning workshop, and the process control device can include a temperature control system and an auxiliary material transport pipe. The temperature control system includes a melt distribution pipe, the outer layer of which is covered with a heat-insulating layer. The melt distribution pipe is used to transport the melt generated by the final polycondensation reaction kettle from the pipe to a metering pump, and then to distribute it uniformly to the spinning components of the spinning box for processing. Since the melt distribution pipe needs heat-insulating treatment, upon detection, the image of the melt distribution pipe can be collected to determine whether the heat-insulating layer is damaged, and the temperature of the melt distribution pipe can be ensured. Among them, the auxiliary material transport pipe is used for transporting industrial auxiliary materials.

[0024] The temperature control system further includes a heating device, which is required to keep the safety gate closed during operation.

[0025] In S102, a first image feature is extracted from the image to be processed.

[0026] In S103, the first image feature is processed based on a decoder network to obtain a fault detection result for the process control device.

[0027] Wherein, the decoder network includes a plurality of decoder modules connected in series in sequence. For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used for classifying and predicting the output features of the decoder layer to obtain a first fault classification result, and updating the first fault classification result according to a preset classification calibration matrix to obtain a second fault classification result output by the decoder module. The fault detection result output by the decoder network includes the second fault classification result output by the decoder module of the last layer of the decoder network and the fault location output by the decoder module of the last layer.

[0028] Here, the decoder network indicates n decoder modules, as shown in FIG. 2, where n is a positive integer. In FIG. 2, the decoder module 1, the decoder module 2, ..., the decoder module n are connected in series in order. The decoder module 1 includes a decoder layer 1 and an adaptive classification head 1, and each decoder layer may include a self-attention module and a cross-attention module (not shown) connected after the self-attention module. Each decoder module after the first decoder module in the decoder network constructs the required query vector (Q), key vector (K) and value vector (V) in the decoder module based on the output features output by the decoder module immediately before the decoder module. Below the decoder layer 1, the adaptive classification head 1 is connected in series, which is used to classify and predict the output features of the decoder layer 1 to obtain a first fault classification result, and update the first fault classification result based on a preset classification calibration matrix to obtain a second fault classification result output by the decoder module. That is, the configuration of each decoder module is basically the same, and each decoder module obtains a first fault classification result and a second fault classification result.

[0029] When the processing of all the n decoder modules is completed, a second fault classification result is obtained, which is output by the decoder module in the last layer of the decoder network.

number

[0030]

number

[0031] In an embodiment of the present disclosure, the decoder network is constructed based on a Transformer network structure.

[0032] In an embodiment of the present disclosure, the decoder network includes a plurality of decoder modules connected in series in order, each decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result. The method of using the adaptive classification head aims to calibrate the prediction result of each layer based on the adaptive classification head to improve the detection performance. In an embodiment of the present disclosure, an image is collected on the back side of a spinning box in a spinning workshop to obtain an image to be processed, and a first image feature is extracted from the image to be processed, and the first image feature is processed based on the decoder network to obtain a fault detection result for a process control device, which method is useful for automating the monitoring of the spinning box in a spinning workshop and timely finding the fault area and fault type.

[0033] The overall flow of the embodiment of the present disclosure can be divided into three parts: obtaining first image features, optimizing the decoder network, and obtaining fault detection results. These three parts are described in detail below.

[0034] 1) First image feature acquisition

[0035] In some embodiments, extracting first image features from the target image may be performed as follows.

[0036] In step A1, an image to be processed is input to the backbone network to obtain initial image features.

[0037] Examples of the backbone network that can be adopted include vggnet (Visual Geometry Group Net, a computer vision network), resnet (residual network), and inception (start-up) network. Of course, the embodiments of the present disclosure are not limited to a specific backbone network.

[0038] In step A2, the initial image feature is input to an encoder network to obtain the first image feature.

[0039] Here, the encoder network may be a Transformer encoder.

number

[0040] In the embodiment of the present disclosure, the backbone network and the encoder network are used to perform feature extraction on the processed image, and the first image features for fault detection can be accurately obtained.

[0041] In some embodiments, as mentioned above, since the temperature control of the thermal insulation layer is important to the spinning process, in the embodiments of the present disclosure, when a thermal image of the spinning workshop is collected based on an infrared camera, extracting the first image feature from the image to be processed can be further implemented as follows:

[0042] In step B1, the image to be processed is input to the segment-anything model to obtain a mask image of the target position.

[0043] Here, the image to be processed is input to the segment anything model, and the image to be processed is input to the image feature extraction network to obtain the encoding feature, as shown in FIG. 3. At least one convolution layer is used to perform convolution processing on the preset mask to obtain the mask feature. Here, the preset mask may be a mask based on points or frames, or may be a mask obtained by segmenting an object based on another segmentation network. The mask feature and the encoding feature are fused and then input to a mask decoder. At the same time, the presentation information (including at least one of points, frames, and text) is also input to the mask decoder. After processing by the mask decoder, a mask image of the target position can be obtained. The target position in the embodiment of the present disclosure may be a damaged position in the thermal insulation layer.

[0044] During implementation, image samples of the damaged position of the thermal insulation layer can be collected in advance. A feature of the damaged position is constructed based on the image samples. For example, a statistical analysis can be performed on the pixel values ​​of the damaged position to obtain a distribution of pixel values ​​as a feature of the damaged position. The distribution of pixel values ​​can be expressed by the pixel amount of different pixel value intervals. Furthermore, for example, feature extraction can be performed on the damaged position to obtain the feature of the damaged position.

[0045] In the embodiment of the present disclosure, images are collected at fixed points of a designated patrol inspection route by controlling a drone. That is, the contents of consecutively collected images of the same detection area in the process control device of the spinning box are almost similar. As a result, in the embodiment of the present disclosure, multiple candidate detection frames can be pre-marked on the images to be processed that are collected for the same detection area. For each image to be processed, pixel points or pixel frames are selected from each candidate detection frame based on the characteristics of the damage position, and the presentation information of the segment-anything model is constructed.

[0046] Specifically, when the feature of the damage location is a pixel value distribution, obtain a number of pixel points within m neighborhoods from any point in the candidate detection frame, and then statistically obtain the pixel distribution to be compared, match the pixel distribution to be compared with the feature of the damage location, calculate the matching degree, and if the matching degree is higher than a threshold, obtain the point and / or frame in the presentation information of the segment-anything model, where m is a positive integer.

[0047] When obtaining the features of the damage location by feature extraction, feature extraction is performed on the candidate detection frame to obtain the features to be compared, and the features to be compared are matched with the features of the damage location. If the degree of matching is higher than a threshold, a point and / or frame in the presentation information of the segment-anything model is obtained.

[0048] In step B2, based on the mask image of the target position, a minimum rectangular bounding box is generated so as to contain the mask image of the target position.

[0049] The points and frames in the pre-constructed presentation can be understood as roughly determining the fracture location. With the help of the presentation, the segment-anything model can be used to improve and optimize the fracture location, thereby obtaining a more accurate mask image.

[0050] In order to improve the accuracy of fault detection, the embodiment of the present disclosure describes the characteristics of the target position from multiple aspects, which can be specifically shown as steps B3 to B7.

[0051] In step B3, a visible sub-image is extracted from the image to be processed based on a minimum rectangular bounding box, and a thermal sub-image in the minimum rectangular bounding box is extracted from a thermal image having the same view range as the image to be processed.

[0052] In step B4, the visible sub-image is input into the backbone network to obtain a first sub-image feature.

[0053] In step B5, the first sub-image feature is input to an encoder network to obtain a sub-image feature to be fused.

[0054] In step B6, thermal features are extracted from the thermal sub-images.

[0055] Note that the execution timing of step B6 only needs to be reasonable, and is not limited to being executed after step B5.

[0056] Here, a long short-term memory (LSTM) network can be used to process the thermal sub-images and obtain thermal features. For example, a drone can be controlled to follow the same patrol inspection route and collect images at fixed points. The view ranges when obtaining images are almost the same. Therefore, in multiple patrol inspections, the objects contained in multiple images at the same position point are almost the same. The image contents in the minimum rectangular bounding box in multiple thermal images within a preset period are obtained to obtain a sequence of thermal sub-images, and the temperature change trend features are extracted based on LSTM to be used as thermal features.

[0057] In step B7, the sub-image feature to be fused, the thermal feature, and the second sub-image feature extracted by the segment-anything model from the minimum rectangular bounding box of the image to be processed are fused to obtain a first image feature.

[0058] Here, the fusion method may be weighted fusion, splice fusion, etc.

[0059] In the embodiment of the present disclosure, the segment-anything model can be used to process the image to be processed, obtain a mask image of the target location, and further obtain a visible sub-image. The visible sub-image is input to a backbone network and an encoder network to obtain a sub-image feature to be fused, and further obtain a thermal sub-image of the visible sub-image based on the visible sub-image, and then extract the thermal feature of the thermal sub-image, and fuse the sub-image feature to be fused, the thermal feature, and the second sub-image feature extracted by the segment-anything model from the minimum rectangular bounding box of the image to be processed to obtain a first image feature. The first image feature can be extracted from multiple dimensions based on this method. In particular, the thermal feature can describe the temperature change trend of the target location, which lays a strong foundation for obtaining subsequent fault detection results.

[0060] 2) Optimization of the decoder network

[0061] In order to further improve the detection accuracy, a query ordering layer can also be added based on FIG. 2 to optimize the decoder network. As shown in FIG. 4, the decoder module 1, the decoder module 2, ..., the decoder module n are connected in series in order. Except for the decoder module of the last layer, at least one other decoder module includes a decoder layer, an adaptive classification head, and a query ordering layer. The adaptive classification head 1 is connected in series under the decoder layer 1 in the decoder module, the query ordering layer 1 is connected in series under the adaptive classification head 1, and the configuration in the decoder module 2 is connected in series under in this manner. Each decoder layer can include a self-attention module and a cross-attention module (not shown) connected after the self-attention module. If the decoder network includes n decoder modules, the query vector is iteratively adjusted n times.

number

number

[0062] The first sub-feature is used to capture semantic information, and the second sub-feature is used to encode fault location information, such as bounding box center and size distribution.

[0063] In the following, we will explain the method of updating two sub-features in the query vector, which specifically includes the following:

[0064] 1) Update the first sub-feature

[0065] As described above, the query vector of each decoder module includes a first sub-feature. At least one decoder module in the decoder network, except for the first decoder module, further includes a query ordering layer. For a decoder module that includes a query ordering layer, the decoder module updates the first sub-feature of the decoder module according to the following method:

[0066] In step C1, according to the second fault classification result output by the decoder module, a first element in a first sub-feature of the decoder module is descended to obtain a first intermediate feature.

[0067] Here, the first intermediate feature of the L-1th layer decoder module is shown in, for example, equation (2).

number

[0068] Each decoder module not only outputs a second fault classification result, but also outputs w detection windows, where w is a positive integer. Accordingly, the first sub-feature includes w elements, and each element in the first sub-feature corresponds to one detection window. L-1 Similarly, includes w elements, each element corresponds to a fault classification result of one detection window, there are m preset fault types, and the fault classification result of any one of the detection windows is a probability distribution {m1, m2, ..., mm} for each preset fault type, where m1 represents the probability that the detection window belongs to fault type 1, and m2 represents the probability that the detection window belongs to fault type 2. The elements in the first sub-feature are ordered in descending order using the maximum probability in the fault classification result of the detection window.

[0069] In step C2, the first intermediate feature is stitched with a pre-defined content calibration matrix to obtain a first stitched feature.

[0070] In step C3, the first spliced ​​feature is processed based on a fully connected layer to obtain a first sub-feature required for the next decoder module of the decoder module.

[0071] After obtaining the first intermediate feature of the L-1th layer decoder module, the first sub-feature of the next decoder module is, for example, as shown in Equation (3).

number

[0072] In an embodiment of the present disclosure, the first intermediate feature of the previous layer is ordered using the second fault classification result of the previous layer, and the first sub-feature of the layer is adjusted based on the first intermediate feature and a preset content calibration matrix, thereby optimizing the ordering to obtain a first sub-feature required by the decoder module of the layer that is more accurately described.

[0073] As described above, the query vector further includes a second sub-feature for encoding fault location information, and the first sub-feature and the second sub-feature have the same number of elements.

[0074] For a decoder module including the query ordering layer, ordering or restructuring the second sub-features based on the different DETR-based detectors for the second sub-features to match an order of the second sub-features with the ranked first sub-features.

[0075] In one possible embodiment, for an H-DETR detector (hybrid matching approach), by using the same second sub-features for the decoder modules of all layers, the decoder modules simply order the second sub-features of the previous layer, and update the second sub-features of the decoder modules based on the following method:

[0076] The update scheme of the second sub-feature of the decoder module is shown, for example, in equation (4).

number

[0077] In implementation, taking w detection windows as an example, the second sub-feature of the L-1th layer decoder module includes w elements, and each element in the second sub-feature of the L-1th layer decoder module corresponds to one detection window. L-1 Similarly, includes w elements, each element corresponds to a fault classification result of one detection window, and there are m preset fault types. The fault classification result of any one of the detection windows is a probability distribution {m1, m2, ..., mm} for each preset fault type, where m1 represents the probability that the detection window belongs to fault type 1, and m2 represents the probability that the detection window belongs to fault type 2. Using the maximum probability in the fault classification result of the detection window, the elements corresponding to each detection window in the second sub-feature of the decoder module of the L-1th layer are ordered in descending order to obtain the second sub-feature of the decoder module of the Lth layer.

[0078] In another possible embodiment, for a DINO-DETR detector (DETR with Improved deNoising anchor boxes), the decoder module updates the second sub-feature of the decoder module based on the following method.

[0079] In step D1, according to a second fault classification result output by the decoder module, the second sub-features of the decoder module are descended to obtain second intermediate features.

[0080] Here, the second intermediate feature of the L-th layer decoder module is shown in, for example, equation (5).

number

[0081] When implemented, the specific ordering scheme is similar to the ordering scheme of the first sub-feature, and the embodiments of the present disclosure will not repeat it.

[0082] In step D2, a second intermediate feature is reconstructed based on the sinusoidal position-encoding function and the multi-layer perceptron to obtain a second sub-feature required for the next decoder module of the decoder module.

[0083] The second sub-characteristic of the L-th layer decoder module is shown, for example, in equation (6).

number

[0084]

number

[0085] In equation (7), (u,v) represent the coordinate position of each point in the detection frame in the image to be processed, D represents the encoding dimension required for the second sub-feature, where D is typically a predetermined value, and T is the modulation temperature, where 10,000 is typically preferred.

[0086] After the position encoding in Equation (7), the updated second sub-feature is obtained by multi-layer perceptron processing.

[0087] In the embodiment of the present disclosure, the second intermediate feature is reconstructed based on the sinusoidal position encoding function and the multi-layer perceptron to obtain the second sub-feature required for the next decoder module, which can more accurately describe the detection window of the fault object and facilitate subsequent processing.

[0088] In some embodiments, the use of a trained decoder network in the embodiments of the present disclosure has been described. In order to obtain a suitable decoder network, in the training process of the decoder network, in the embodiments of the present disclosure, the classification calibration matrix of each decoder module in the decoder network can be learned and obtained by training the decoder network. It can be implemented as follows:

[0089] In the process of training the decoder network, since the control of the melt temperature in the spinning workshop is relatively important, and the fault identification focuses on evaluating the thermal insulation effect of the thermal insulation layer, the classification calibration matrix may be initialized randomly, which is far from the ideal state. In order to accelerate the convergence of the model, the embodiment of the present disclosure can initialize the classification calibration matrix according to the following method:

[0090] In step H1, a sub-sample of the fault area is obtained by dividing a sample of thermal images collected for the process control device.

[0091] In step H2, the classification calibration vector is initialized based on the difference between the sub-samples and sample data when the failure area is in a normal operating state.

[0092] In the embodiment of the present disclosure, the classification calibration vector is initialized based on the difference between the sub-sample and the sample data in the fault area under normal operating conditions, so that the initialization state of the classification calibration vector can be expressed based on the temperature information of the thermal insulation layer, which facilitates accelerating the convergence of model training and helps to learn the available classification calibration vector as quickly as possible.

[0093] 3) Obtaining fault detection results

[0094] In some embodiments, the process control device includes a temperature control system as described above, the temperature control system including a heating device and a melt delivery pipe. Temperature sensing can be implemented as follows.

[0095] In step E1, a thermal image of the temperature control system is obtained based on an infrared camera.

[0096] In step E2, for a target component of the temperature control system, an image block of the target component is extracted from the thermal image.

[0097] Here, the target part may be a melt distribution pipe or a heating device.

[0098] In step E3, difference information between the image block of the target part and the standard image block of the target part is obtained by comparing the image block of the target part and the standard image block of the target part.

[0099] The standard image block of the target part is the standard temperature range of the target part. The collected image block of the target part is compared with the standard temperature range of the target image to obtain difference information between the two. Specifically, the difference information can be expressed based on statistical information of the difference in pixel values ​​at the same pixel position. The statistical information is, for example, the average value, the variance, etc. Also, the feature matrices of the two can be extracted respectively using a pre-trained model, and then the difference between the feature matrices can be calculated to obtain the difference information.

[0100] In step E4, it is determined whether the temperature of the target component is within a normal temperature range based on the difference information.

[0101] If the image block of the target part belongs to the standard temperature range of the target image, it is determined that the target part is not abnormal, whereas if the image block of the target part does not belong to the standard temperature range of the target image, it is determined that the target part is abnormal. The second fault classification result output by the decoder module in the last layer of the decoder network in the fault detection result and the fault location output by the decoder module in the last layer are combined to determine the abnormal situation.

[0102] In an embodiment of the present disclosure, the captured thermal image of the target part is compared to a standard image block of the target part to determine whether the temperature of the target part is normal at that time.

[0103] In some embodiments, the second fault classification result in the fault detection result includes a fault type and a fault grade, and the method further includes:

[0104] In step F1, if the fault detection result indicates that a fault occurs in the temperature control system of the process control device and the fault grade is higher than a preset grade, a repair task is generated, and the repair task includes a repair task urgency and the fault detection result.

[0105] In step F2, the repair task is pushed to the target client.

[0106] Here, for example, if it is detected that the damage of the insulation layer on the upper surface of the molten distribution pipe in the temperature control system is too large (e.g., the depth of the damage is larger than the preset depth and the radius of the damage is larger than the preset radius) and affects the temperature of the insulation pipe (e.g., an abnormality is found based on thermal images), the fault detection result is the highest risk and needs to be dealt with immediately. If it is detected that the damage of the insulation layer on the upper surface of the molten distribution pipe in the temperature control system is too small and does not affect the temperature of the insulation pipe, the fault detection result is low risk, and the repair task can be recorded first and handled regularly by employees.

[0107] In the embodiments of the present disclosure, different handling strategies are implemented based on the urgency of the fault, and can be adaptively adjusted based on the actual situation.

[0108] In some embodiments, image collection is also required for open areas of the spinning workshop, where process auxiliary materials may accumulate, and can be implemented as follows:

[0109] In step G1, an image is collected for an open area of ​​a spinning box in the spinning workshop to obtain a target image.

[0110] In step G2, a deposition state of a process auxiliary material in the open area is detected based on the target image, and a detection result is obtained.

[0111] In step G3, a detection record is generated based on the detection result and historical detection results of the open area.

[0112] In the case of a detection result showing that there is an accumulation of process auxiliary materials at position coordinate 1, since there are already two records in the historical detection results showing "There is an accumulation of process auxiliary materials at position coordinate 1," the detection result is continued to be recorded as a third record showing "There is an accumulation of process auxiliary materials at position coordinate 1."

[0113] In step G4, the detection records are matched with a set of reporting strategies.

[0114] Here, the set of reporting strategies is to report to a first-class employee when there is one identical detection record, to report to a second-class employee when there are two identical detection records, and to report to a q-class employee when there are q identical detection records recorded, and thus analogized. The q-class manager is the senior manager of the q-1-class manager, and thus analogized, the 2-class manager is the senior of the 1-class manager.

[0115] In step G5, alarm information is generated based on the reporting strategies obtained by matching in the set of reporting strategies.

[0116] In the embodiment of the present disclosure, to compensate for the management vulnerability, different reporting strategies are implemented based on the status of the deposition of auxiliary materials in the process to avoid operators forgetting about it.

[0117] To summarize the above, in one possible embodiment, a network framework applicable to the spinning workshop detection method proposed in the embodiments of the present disclosure includes, for example, as shown in Figure 5, inputting an image to be processed into a backbone network to obtain initial image features, inputting the initial image features into an encoder network to obtain the first image features, and inputting the first image features into a decoder network including n decoder modules.

[0118] In the embodiment of the present disclosure, the query vector is initialized as Q 0 Then, the processing image is divided into multiple grids, each grid is position-encoded, and the target position encoding PE is obtained. The initialization query vector Q 0 After the fusion process of and the target position encoding PE, it is set as the query vector Q of the self-attention module 1 in the first decoder module 1, and the target position encoding PE is set as K and V of the self-attention module 1 in the first decoder module 1. The output of the self-attention module 1 is the self-attention feature TE1.

[0119] The self-attention feature TE1 and the target position encoding PE are fused and defined as Q of the cross attention module 1 in the first decoder module 1, the first image feature is defined as V of the cross attention module 1, and the fused feature of the first image feature and a pre-defined position encoding CE at the input of the encoder network (the position encoding is the position encoding of the camera, for example, the position encoding in the camera's patrol inspection path) is defined as K of the cross attention module 1.

[0120] For each subsequent decoder module L, the Q, K, and V of that self-attention module L are constructed based on the target position encoding PE and the updated Q of the previous decoder module. For example, K and V are both target position encoding PE, and Q is the updated Q based on the previous decoder module.

[0121] For each subsequent decoder module L, the Q, K, and V of the cross-attention module L are constructed based on the self-attention features TEL, the first image features, and the preset location encoding. For example, Q is the self-attention features TEL output by the self-attention module L, V is the first image features, and K is the fusion feature of the first image features and the preset location encoding.

[0122] In the process, the final fault detection result and fault location are obtained through the processing of each layer, the decoder layer, the adaptive classification head, and the query ordering layer.

[0123] Based on the same technical idea, an embodiment of the present disclosure provides a detection device 600 for a spinning workshop, as shown in FIG. 6 , A collection unit 601 for performing image collection for a process control device of a spinning box in a spinning workshop to obtain images to be processed; an extraction unit 602 for extracting a first image feature from the image to be processed; a processing unit 603 for processing the first image feature based on the decoder network to obtain a fault detection result for the process control device; The decoder network includes a plurality of decoder modules connected in series, For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result, and update the first fault classification result according to a preset classification calibration matrix to obtain a second fault classification result output by the decoder module; The fault detection result output by the decoder network includes a second fault classification result output by the decoder module in the last layer of the decoder network and a fault location output by the decoder module in the last layer.

[0124] In some embodiments, the query vector of each decoder module includes a first sub-feature for capturing semantic type information; At least one decoder module other than the last decoder module in the decoder network further comprises a query ordering layer; The apparatus comprises: For a decoder module including the query ordering layer, Descending a first element in a first sub-feature of the decoder module according to a second fault classification result output by the decoder module to obtain a first intermediate feature; Concatenating the first intermediate feature with a preset content calibration matrix to obtain a first concatenated feature; The decoder further comprises a first updating unit for updating a first sub-feature of the decoder module based on a method of processing the first spliced ​​feature based on a fully connected layer to obtain a first sub-feature required for a next decoder module of the decoder module.

[0125] In some embodiments, the query vector of each decoder module further includes a second sub-feature for encoding fault location information, and the first sub-feature and the second sub-feature have the same number of elements; The apparatus comprises: For a decoder module including the query ordering layer, According to the second fault classification result output by the decoder module, descending the second sub-feature of the decoder module to obtain a second intermediate feature; The decoder further comprises a second updating unit for updating a second sub-feature of the decoder module based on the method of reconstructing a second intermediate feature based on a sinusoidal position coding function and a multi-layer perceptron to obtain a second sub-feature required for a next decoder module of the decoder module.

[0126] In some embodiments, the second fault classification result in the fault detection result includes a fault type and a fault grade, and the device: generating a repair task including an urgency of the repair task and the fault detection result when the fault detection result indicates that a fault occurs in the temperature control system of the process control device and the fault level is higher than a preset level; and pushing the repair task to a target client side.

[0127] In some embodiments, the extraction unit 602 comprises: inputting the image to be processed into a backbone network to obtain initial image features; inputting the initial image features into an encoder network to obtain the first image features.

[0128] In some embodiments, the extraction unit 602 comprises: inputting the image to be processed into a segment-anything model to obtain a mask image of a target position; generating a minimum rectangular bounding box based on the mask image of the target position so as to contain the mask image of the target position; extracting a visible sub-image from the image to be processed based on the minimum rectangular bounding box, and extracting a thermal sub-image in the minimum rectangular bounding box from a thermal image having the same view range as the image to be processed; inputting the visible sub-image into a backbone network to obtain a first sub-image feature; inputting the first sub-image feature into an encoder network to obtain a sub-image feature to be fused; extracting thermal features from the thermal sub-images; fusing the sub-image feature to be fused, the thermal feature, and a second sub-image feature extracted by the segment-anything model from the minimum rectangular bounding box of the image to be processed to obtain the first image feature.

[0129] In some embodiments, the process control device comprises a temperature control system, the device comprising: acquiring a thermal image of the temperature control system based on an infrared camera; For a target component of the temperature control system, extracting an image block of the target component from the thermal image; obtaining difference information between the image block of the target part and the standard image block of the target part by comparing the image block of the target part with the standard image block of the target part; and detecting whether the temperature of the target component is within a normal temperature range based on the difference information.

[0130] In some embodiments, the apparatus further comprises: Performing image acquisition on an open area of ​​a spinning box in the spinning workshop to obtain a target image; Detecting a deposition status of a process auxiliary material in the open area based on the target image to obtain a detection result; generating a detection record based on the detection results and historical detection results of the open area; matching the detection records with a set of reporting strategies; and generating alarm information based on the matching reporting strategies in the set of reporting strategies.

[0131] In some embodiments, the classification calibration matrix of each decoder module in the decoder network is learnable and is obtained by training the decoder network, and the apparatus further comprises: In the process of training the decoder network, Dividing out a sub-sample of a fault area from a sample of thermal images collected for the process control device; The classification calibration matrix may further include an initialization unit for initializing the classification calibration vector according to a method in which the classification calibration matrix is ​​initialized according to a difference between the sub-sample and sample data in a normal operating state of the failure area.

[0132] The specific functions and exemplary descriptions of each module, sub-module / unit of the apparatus according to the embodiments of the present disclosure may refer to the relevant descriptions of the corresponding steps in the above-mentioned method embodiments, and will not be repeated here.

[0133] In the technical solution disclosed herein, the acquisition, storage, and application of users' personal information comply with the provisions of relevant laws and regulations and do not violate public order and morals.

[0134] FIG. 7 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 7, the electronic device includes a memory 710 and a processor 720, and the memory 710 stores a computer program executable by the processor 720. The number of the memory 710 and the processor 720 may be one or more. The memory 710 may store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided by the above method embodiment. The electronic device may further include: The communication interface 730 is used for communicating with an external device and performing data interaction and transmission.

[0135] When the memory 710, the processor 720, and the communication interface 730 are implemented independently, the memory 710, the processor 720, and the communication interface 730 are connected to each other via a bus and can communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be classified into an address bus, a data bus, a control bus, and the like. For ease of explanation, only one thick line is shown in FIG. 7, but this does not indicate only one bus or only one type of bus.

[0136] Optionally, in a specific implementation, when the memory 710, the processor 720, and the communication interface 730 are integrated on one chip, the memory 710, the processor 720, and the communication interface 730 can communicate with each other via an internal interface.

[0137] It should be understood that the processor may be a Central Processing Unit (CPU) or may be other general purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware assemblies, etc. The general purpose processor may be a microprocessor or any conventional processor, etc. Additionally, the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0138] Additionally, optionally, the memory may include read-only memory and random access memory, and may further include non-volatile random access memory. The memory may be either volatile or non-volatile memory, or may include both volatile and non-volatile memory, where non-volatile memory may include Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), or flash memory. Volatile memory may include Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (SLDRAM), and direct RAMBUS RAM (DR RAM).

[0139] The above-mentioned embodiments may be implemented in whole or in part in software, hardware, firmware, or any combination thereof. When implemented in software, the whole or in part may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, a process or function according to the embodiments of the present disclosure is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)). Note that the computer-readable storage medium referred to in this disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0140] Those skilled in the art can understand that all or part of the steps for realizing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0141] In the description of the embodiments of the present disclosure, the reference words "one embodiment", "several embodiments", "examples", "specific examples", or "several examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. And, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. Furthermore, a person skilled in the art may combine different embodiments or examples described in the present disclosure and features of different embodiments or examples to the extent that they are not mutually inconsistent.

[0142] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean either A or B. "And / or" in the present disclosure is merely for describing the related relationship of related objects, and indicates that three kinds of relationships may exist, for example, A and / or B can indicate the following three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0143] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying a relative importance, nor should they be construed as implying a number of technical features shown. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "plurality" means two or more, unless otherwise specified.

[0144] The above are merely illustrative examples of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. A method for detecting a spinning workplace, comprising the steps of: performing image acquisition on a process control device of a spinning box in a spinning workshop to obtain images to be processed; Extracting a first image feature from the target image; processing the first image features based on a decoder network to obtain a fault detection result for the process control device; The decoder network includes a plurality of decoder modules connected in series in series; For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result, and update the first fault classification result based on a preset classification calibration matrix to obtain a second fault classification result output by the decoder module; The fault detection result output by the decoder network includes a second fault classification result output by a decoder module in a last layer of the decoder network, and a fault location output by a decoder module in the last layer; for each decoder module after a first decoder module in the decoder network, the output features of the decoder module immediately preceding the decoder module are taken as the input features of the decoder module; Method for detecting spinning workplaces.

2. The query vector of each decoder module includes a first sub-feature for capturing semantic type information; At least one decoder module other than the last decoder module in the decoder network further comprises a query ordering layer; For a decoder module including the query ordering layer, the decoder module further comprises: Descending a first element in a first sub-feature of the decoder module based on a second fault classification result output by the decoder module to obtain a first intermediate feature; Concatenating the first intermediate feature with a preset content calibration matrix to obtain a first concatenated feature; Processing the first spliced ​​feature based on a fully connected layer to obtain a first sub-feature required for a next decoder module of the decoder module; updating the first sub-feature of the decoder module based on the method; The method for detecting a spinning workplace according to claim 1.

3. The query vector of each decoder module further includes a second sub-feature for encoding fault location information, and the first sub-feature and the second sub-feature have the same number of elements; a decoder module including the query ordering layer, Descending the second sub-feature of the decoder module according to the second fault classification result output by the decoder module to obtain a second intermediate feature; reconstructing a second intermediate feature based on a sinusoidal position-encoding function and a multi-layer perceptron to obtain a second sub-feature required for a next decoder module of the decoder module; and updating the second sub-feature of the decoder module based on the method; The method for detecting a spinning workplace according to claim 2.

4. The second fault classification result in the fault detection result includes a fault type and a fault grade, The method for detecting a spinning workplace includes: generating a repair task including an urgency of the repair task and the fault detection result when the fault detection result indicates that a fault occurs in the temperature control system of the process control device and the fault level is higher than a preset level; and pushing the repair task to a target client. The method for detecting a spinning workplace according to claim 1.

5. Extracting a first image feature from the target image includes: inputting the image to be processed into a backbone network to obtain initial image features; inputting the initial image features into an encoder network to obtain the first image features; The method for detecting a spinning workplace according to claim 1.

6. Extracting a first image feature from the target image includes: inputting the image to be processed into a segment anything model to obtain a mask image of a target position; generating a minimum rectangular bounding box based on the mask image of the target position so as to contain the mask image of the target position; extracting a visible sub-image from the image to be processed based on the minimum rectangular bounding box, and extracting a thermal sub-image in the minimum rectangular bounding box from a thermal image having the same view range as the image to be processed; inputting the visible sub-image into a backbone network to obtain a first sub-image feature; inputting the first sub-image feature into an encoder network to obtain a sub-image feature to be fused; extracting thermal features from the thermal sub-images; fusing the sub-image feature to be fused, the thermal feature, and a second sub-image feature extracted by the segment anything model from the minimum rectangular bounding box of the image to be processed to obtain the first image feature; The method for detecting a spinning workplace according to claim 1.

7. the process control device includes a temperature control system; The method for detecting a spinning workplace includes: acquiring a thermal image of the temperature control system based on an infrared camera; For a target component of the temperature control system, extracting an image block of the target component from the thermal image; obtaining difference information between the image block of the target part and the standard image block of the target part by comparing the image block of the target part with the standard image block of the target part; detecting whether the temperature of the target component is within a normal temperature range based on the difference information; The method for detecting a spinning workplace according to claim 1.

8. The method for detecting a spinning workplace includes: Performing image acquisition on an open area of ​​a spinning box in the spinning workshop to obtain a target image; Detecting a deposition status of a process auxiliary material in the open area based on the target image to obtain a detection result; generating a detection record based on the detection results and historical detection results of the open area; matching the detection records with a set of reporting strategies; generating alert information based on the matching reporting strategies in the set of reporting strategies; The method for detecting a spinning workplace according to claim 1.

9. A detection device for a spinning workshop, comprising: a collection unit for performing image collection for a process control device of a spinning box in a spinning workshop to obtain images to be processed; an extraction unit for extracting first image features from the target image; a processing unit for processing the first image features based on a decoder network to obtain a fault detection result for the process control device; The decoder network includes a plurality of decoder modules connected in series in series; For each decoder module, the decoder module includes a decoder layer and an adaptive classification head, and the adaptive classification head is used to classify and predict the output features of the decoder layer to obtain a first fault classification result, and update the first fault classification result based on a preset classification calibration matrix to obtain a second fault classification result output by the decoder module; The fault detection result output by the decoder network includes a second fault classification result output by a decoder module in a last layer of the decoder network, and a fault location output by a decoder module in the last layer; for each decoder module after a first decoder module in the decoder network, the output features of the decoder module immediately preceding the decoder module are taken as the input features of the decoder module; Detection device for spinning workshops.

10. The query vector of each decoder module includes a first sub-feature for capturing semantic type information; At least one decoder module other than the last decoder module in the decoder network further comprises a query ordering layer; The detection device of the spinning workshop, for a decoder module including the query ordering layer, Descending a first element in a first sub-feature of the decoder module based on a second fault classification result output by the decoder module to obtain a first intermediate feature; Concatenating the first intermediate feature with a preset content calibration matrix to obtain a first concatenated feature; a first updating unit for updating a first sub-feature of the decoder module based on a method of processing the first spliced ​​feature based on a fully connected layer to obtain a first sub-feature required for a next decoder module of the decoder module; 10. A spinning station detection device according to claim 9.

11. The query vector of each decoder module further includes a second sub-feature for encoding fault location information, and the first sub-feature and the second sub-feature have the same number of elements; The detection device of the spinning workshop, for a decoder module including the query ordering layer, Descending the second sub-feature of the decoder module according to the second fault classification result output by the decoder module to obtain a second intermediate feature; and a second updating unit for updating a second sub-feature of the decoder module based on the method of: reconstructing a second intermediate feature based on a sinusoidal position-encoding function and a multi-layer perceptron to obtain a second sub-feature required for a next decoder module of the decoder module. A spinning station detection device according to claim 10.

12. The second fault classification result in the fault detection result includes a fault type and a fault grade, The detection device in the spinning workshop comprises: generating a repair task including an urgency of the repair task and the fault detection result when the fault detection result indicates that a fault occurs in the temperature control system of the process control device and the fault level is higher than a preset level; and pushing the repair task to a target client side.

10. A spinning station detection device according to claim 9.

13. The extraction unit comprises: inputting the image to be processed into a backbone network to obtain initial image features; inputting the initial image features into an encoder network to obtain the first image features; 10. A spinning station detection device according to claim 9.

14. The extraction unit comprises: inputting the image to be processed into a segment anything model to obtain a mask image of a target position; generating a minimum rectangular bounding box based on the mask image of the target position so as to contain the mask image of the target position; extracting a visible sub-image from the image to be processed based on the minimum rectangular bounding box, and extracting a thermal sub-image in the minimum rectangular bounding box from a thermal image having the same view range as the image to be processed; inputting the visible sub-image into a backbone network to obtain a first sub-image feature; inputting the first sub-image feature into an encoder network to obtain a sub-image feature to be fused; extracting thermal features from the thermal sub-images; fusing the sub-image feature of the fusion target, the thermal feature, and a second sub-image feature extracted by the segment anything model from the minimum rectangular bounding box of the image of the target to obtain the first image feature; 10. A spinning station detection device according to claim 9.

15. the process control device includes a temperature control system; The detection device in the spinning workshop comprises: acquiring a thermal image of the temperature control system based on an infrared camera; For a target component of the temperature control system, extracting an image block of the target component from the thermal image; obtaining difference information between the image block of the target part and the standard image block of the target part by comparing the image block of the target part with the standard image block of the target part; and detecting whether the temperature of the target component is within a normal temperature range based on the difference information.

10. A spinning station detection device according to claim 9.

16. The detection device in the spinning workshop comprises: Performing image acquisition on an open area of ​​a spinning box in the spinning workshop to obtain a target image; Detecting a deposition status of a process auxiliary material in the open area based on the target image to obtain a detection result; generating a detection record based on the detection results and historical detection results of the open area; matching the detection records with a set of reporting strategies; and generating alarm information based on the matching report strategies in the set of report strategies.

10. A spinning station detection device according to claim 9.

17. At least one processor; a memory in communication with the at least one processor; An electronic device, the memory storing instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 8.

Images