Scanned image processing method, device, equipment, storage medium and product

Abstract

The present application discloses a method, apparatus, device, storage medium, and product for processing scanned images, relating to the field of image processing technology. Since a scanned image has the characteristic of being obtained by sequentially scanning its pixel regions, the present application uses the self-attention mechanism of the Transformer architecture decoder to extract the correlation between visual features based on the visual features extracted by the visual feature extractor. This allows the second feature sequence extracted by the feature extraction module to fully exploit the spatiotemporal correlations in the visual features. This improves the accuracy of task processing when applied to downstream scanned image processing tasks.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a scanned image processing method, device, equipment, storage medium and product. Background Art

[0002] Among related technologies, scanning probe microscopy (SPM) technology is widely used in materials science, biology, chemistry and other fields, especially in the study of surface morphology and material properties at the nanoscale.

[0003] However, current methods for processing scanned images obtained by scanning probes are mostly based on traditional image processing techniques. These techniques typically rely on single features within a single frame or image sequence, while ignoring the deeper connections between features during the scanning process. As a result, these techniques fail to meet the accuracy requirements for processing related tasks involving scanned images.

[0004] Therefore, how to improve the image processing accuracy of scanned images is a problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of this application is to provide a scanned image processing method, device, equipment, storage medium and product, aiming to solve the technical problem of how to improve the image processing accuracy of scanned images.

[0006] To achieve the above objectives, the present application proposes a scanned image processing method, which includes:

[0007] Acquire a scanned image; the scanned image includes a plurality of scan segments obtained by sequential scanning;

[0008] Extracting features from the scanned image using a feature extraction module of a scanned image processing model; the feature extraction module includes a visual feature extractor and a Transformer decoder; the visual feature extractor is configured to extract visual features from the scanned image to obtain a first feature sequence, the first feature sequence including visual features corresponding one-to-one to the plurality of scan segments; the Transformer decoder is configured to capture spatiotemporal correlations between the visual features in the first feature sequence to obtain a second feature sequence;

[0009] The task processing module of the scanning image processing model performs preset task processing on the second feature sequence to obtain a scanning image processing result.

[0010] In some embodiments, before acquiring the scanned image, the scanned image processing method further includes:

[0011] Inputting a sample scan image into an initial feature extraction module to obtain a second feature sequence sample output by the initial feature extraction module; the sample scan image includes a plurality of sample scan segments obtained by sequential scanning, and the initial feature extraction module includes the visual feature extractor and the Transformer decoder; the second feature sequence sample includes sample image features corresponding one-to-one to the plurality of sample scan segments, and the last sample image feature in the second feature sequence sample is a visual feature predicted for the next scan segment of the corresponding sample scan segment;

[0012] Obtaining a predicted image according to the features of the last sample image;

[0013] Calculating the loss between the predicted image and the real image data of the next scanning segment;

[0014] Based on the loss, the parameters of the initial feature extraction module are adjusted to train the feature extraction module.

[0015] In some embodiments, calculating the loss between the predicted image and the real image data of the next scanning segment includes:

[0016] Calculate the loss by expression 1;

[0017] The expression 1 is: ;

[0018] Wherein, L is the loss, is the predicted image, is the real image data.

[0019] In some embodiments, before acquiring the scanned image, the scanned image processing method further includes:

[0020] The visual feature extractor is pre-trained using an image sample set related to the task application domain.

[0021] In some embodiments, pre-training the visual feature extractor includes:

[0022] Pre-train the visual feature extractor using at least one of an autoencoder model, contrastive learning, and a masked autoencoder.

[0023] In some embodiments, the image sample set is an unlabeled image sample set.

[0024] In some embodiments, the task processing module is a probe quality detection module; the task processing module of the scanning image processing model performs preset task processing on the second feature sequence to obtain a scanning image processing result, including:

[0025] Performing pooling processing on the second feature sequence to obtain a one-dimensional feature vector sequence;

[0026] The one-dimensional feature vector sequence is input into a classification model in the probe quality detection module, and based on an output of the classification model, a probe quality assessment result corresponding to the scanned image is determined.

[0027] In addition, to achieve the above-mentioned purpose, the present application also proposes a scanning image processing device, which includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the computer program is configured to implement the steps of the scanning image processing method described above.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the scanning image processing method described above are implemented.

[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the scanning image processing method described above are implemented.

[0030] One or more technical solutions proposed in this application have at least the following technical effects:

[0031] Since scanned images are characterized by sequential scanning of their pixel areas, the feature extraction module, based on the visual features extracted by the visual feature extractor, uses the self-attention mechanism of the decoder of the Transformer architecture to extract the correlation between visual features. This enables the second feature sequence extracted by the feature extraction module to fully explore the spatiotemporal correlation in the visual features. This improves the accuracy of task processing when applied to downstream scanned image processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1A schematic diagram showing a flow chart of a scanned image processing method provided in an embodiment of the present application is shown;

[0035] Figure 2 A schematic diagram of a scanning image acquisition trajectory provided by an exemplary embodiment of the present application is shown;

[0036] Figure 3 A schematic diagram showing channel data contained in a scanned image provided by an exemplary embodiment of the present application is shown;

[0037] Figure 4 A schematic diagram showing an abnormal image before and after processing provided by an exemplary embodiment of the present application is shown;

[0038] Figure 5 Schematic diagrams showing the missing region before and after processing provided by an exemplary embodiment of the present application are shown;

[0039] Figure 6 Schematic diagrams before and after normalization processing provided by an exemplary embodiment of the present application are shown;

[0040] Figure 7 A schematic diagram showing a partial training process of a scanning image processing model provided by an exemplary embodiment of the present application is shown;

[0041] Figure 8 A schematic structural diagram of a scanning image processing device provided in one embodiment of the present application is shown.

[0042] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0044] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0045] The main solution of the embodiment of the present application is: obtaining a scanned image; the scanned image includes multiple scan segments obtained by scanning in sequence; performing feature extraction on the scanned image through the feature extraction module of the scanned image processing model; the feature extraction module includes a visual feature extractor and a Transformer decoder, the visual feature extractor is used to extract visual features of the scanned image to obtain a first feature sequence, the first feature sequence includes visual features that correspond one-to-one to multiple scan segments; the Transformer decoder is used to capture the spatiotemporal correlation between each visual feature in the first feature sequence to obtain a second feature sequence; the task processing module of the scanned image processing model performs preset task processing on the second feature sequence to obtain a scanned image processing result.

[0046] Among related technologies, scanning probe microscopy (SPM) technology is widely used in materials science, biology, chemistry and other fields, especially in the study of surface morphology and material properties at the nanoscale.

[0047] However, current methods for processing scanned images obtained by scanning probes are mostly based on traditional image processing techniques. These techniques typically rely on single features within a single frame or image sequence, while ignoring the deeper connections between features during the scanning process. As a result, these techniques fail to meet the accuracy requirements for processing related tasks involving scanned images.

[0048] Therefore, how to improve the image processing accuracy of scanned images is a problem that needs to be solved urgently.

[0049] Based on this, the present application provides a solution that allows the temporal and spatial correlations between each scan segment to be deeply extracted based on the original characteristics of the scanned image during model training, thereby achieving better efficiency and accuracy when processing downstream tasks with temporal and spatial dependencies.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a scanning image processing device capable of performing the above functions. The following uses a scanning image processing device as an example to illustrate this embodiment and the following embodiments.

[0051] Reference Figure 1 , Figure 1 The flowchart of a scanning image processing method provided by an embodiment of the present application is shown. The scanning image processing method can be applied to a scanning image processing device, and includes the following steps S110 to S130:

[0052] Step S110: acquiring a scanned image.

[0053] Image capture methods generally include photographic imaging and scanning imaging. When photographic imaging is used, a complete image is typically acquired with a single capture, with each pixel captured simultaneously. Scanning imaging, on the other hand, requires scanning different areas of the subject in a specific order to gradually form a scanned image. In other words, the scanned image described in this embodiment refers to an image obtained through scanning imaging.

[0054] For ease of understanding, the scanning probe system is used as an example to explain the scanning image processing method provided in this embodiment. The scanning probe system refers to a microscope system that scans a sample to be tested using a scanning probe to obtain a scanned image. Figure 2 As shown, when the scanning probe is actually scanning, it can be Figure 2 The scanning is performed in the order shown by the arrows, and a complete scanned image is finally obtained. That is, the scanned image may include multiple scan segments obtained by scanning in sequence.

[0055] In this embodiment, the scanned image can be expressed as H×W×C, where H refers to the number of pixel rows of the scanned image, W refers to the number of pixel columns of the scanned image, and C refers to the number of channels corresponding to the scanned image. It should be noted that when the scanning probe scans the sample to be tested, its channels are different depending on the type of scanning probe system, that is, the recorded information is different. As an example, the scanned image collected by the scanning tunneling microscope can contain data of three channels: height, current, and phase (for example Figure 3 In the process of acquiring the scanned image, the scanning may be performed pixel by pixel, row by row, or column by column, which is not limited in this embodiment.

[0056] In this embodiment, each row of pixels can be defined as a scanning segment. In some other feasible implementations, each scanning segment can also be a portion of a row of pixels or multiple rows of pixels. For ease of understanding, the following explanation is based on each row as a scanning segment.

[0057] As mentioned above, since the scanning probe scans line by line, for the scanning probe system, each scanning segment has a strong temporal and spatial correlation with the scanning segments before and after it. This correlation is specifically based on three dimensions: (1) The sample to be tested itself shows correlation, which can be roughly understood as a common picture, and the parts of the object in the picture can have correlations; (2) There is correlation between similar scanning results. This is because the scanning process is carried out line by line, and the results of each line are closely related to the current state of the scanning probe system. (3) Usually, the scanning probe system records information from multiple channels at the same time. The data between these channels are correlated, representing the state of the current system or the sample to be tested.

[0058] It is understandable that, in other scanning imaging technology fields except the scanning probe system, the scanned images obtained therefrom all have the characteristic of temporal and spatial correlation, which will not be elaborated in this embodiment.

[0059] Step S120 , extracting features from the scanned image using a feature extraction module of the scanned image processing model.

[0060] Among them, the feature extraction module includes a visual feature extractor and a Transformer decoder.

[0061] The visual feature extractor is a module used to perform initial feature extraction on a scanned image. It is typically a multi-layer neural network, convolutional neural network, or visual transformer. The visual feature extractor takes the scanned image as input. By extracting visual features from the scanned image, the scanned image is mapped from the image data dimension to the feature dimension, thereby obtaining a first feature sequence corresponding to the scanned image. In this embodiment, the first feature sequence can be a one-dimensional feature sequence.

[0062] The Transformer decoder is a module for further extracting the deep correlation between features. Specifically, the Transformer decoder can be an autoregressive model based on the Transformer architecture, which can be built based on a commonly used encoder-decoder model or a pure decoder model. In this embodiment, the pure decoder model is used as an example to explain the subsequent embodiments.

[0063] After receiving the first feature sequence input by the visual feature extractor, the Transformer decoder can combine the start vector (START vector, used to mark the starting position of the sequence) and capture and fuse the associations between each element through the self-attention mechanism built into the Transformer decoder architecture, thereby outputting a second feature sequence containing spatiotemporal correlations. In the second feature sequence, each element (i.e., feature) contains a spatiotemporal association relationship with other elements.

[0064] It is worth mentioning that the feature extraction module can be pre-trained. The feature extraction module can be trained as a whole, or the visual feature extractor can be pre-trained separately and then used as part of the feature extraction module to train the entire feature extraction module as a whole.

[0065] For ease of understanding, the following explanation is given by taking the example of first pre-training the visual feature extractor separately and then training the entire feature extraction module as a whole.

[0066] Specifically, the visual feature extractor can be pre-trained separately using an image sample set related to the task application field to ensure the accuracy of the visual feature extractor when performing feature extraction in the subsequent overall training process for the entire feature extraction module.

[0067] For the scanning probe scenario, a commonly used convolutional neural network (CNN) can be used to pre-train the CNN skeleton using the image dataset of the scanning probe, thereby increasing the feature recognition capability of the corresponding CNN skeleton for the image data in the scanning probe field. In the process of training the visual feature extractor, some fine-tuning operations are also involved. The relevant parameters of the visual feature extractor can be fine-tuned by one or more of the autoencoder model, contrastive learning, and masked autoencoder to ensure higher feature recognition capabilities. In other feasible implementations, a discrete variational autoencoder (discrete VAE) can also be used to train the visual feature extraction model on the aforementioned scanning probe image dataset, which is not limited in this embodiment.

[0068] It should be noted that, in this embodiment, the image dataset of the scanning probe is obtained from the original sample set after preprocessing, that is, the scanned images in the original sample set have all undergone the process of removing abnormal images, removing incomplete parts, and standardizing and normalizing.

[0069] Specifically, such as Figure 4As shown, for the original sample set, the maximum and minimum values of each scan segment (e.g., each row) in each original sample image in the original sample set can be calculated. Based on the difference between the maximum and minimum values and the mean square error, a comparison is performed using a preset difference threshold and mean square error threshold. If the difference is less than the difference threshold and the mean square error is less than the mean square error threshold, it can be considered that the scan segment in the original sample image is normal. If the difference is not less than the difference threshold or the mean square error is not less than the mean square error threshold, it can be considered that the scan segment in the original sample image is abnormal. If the number of scan segments in the original sample image that have abnormalities exceeds a preset reference ratio, the original sample image is considered to be an abnormal image and can be removed from the original sample set.

[0070] like Figure 5 As shown, for the original sample set, the incomplete parts of each original sample image in the original sample set can also be removed. For example, in actual application scenarios, there are some models of scanning probe systems that may only scan a portion of the sample to be tested. In this case, the scanning probe system of this model may complete the remaining unscanned area (even if no scanning is actually performed). For the incomplete scanned image obtained in this case, the mean square error can be checked one by one according to the scan segment and compared with the pre-set integrity threshold. If the mean square error of the scan segment is less than the integrity threshold, it can be considered that this scan segment is the completed data and can be removed from the original sample image.

[0071] like Figure 6 As shown, after completing Figure 4-5 After the processing shown, the processed original sample image can be standardized and normalized. The maximum and minimum values of the original sample image can be scaled to [0, 1]. This can be achieved by subtracting the minimum value from all pixel values of the original sample image and then dividing by the difference between the maximum and minimum values. The pixel values of the entire original sample image can also be changed to conform to a distribution with a mean of 0 and a standard deviation of 1. Specifically, the mean and variance of all original sample images can be calculated, and then the pixels of each original sample image can be subtracted from the mean and divided by the variance.

[0072] This yields a processed image dataset of the scanning probe (i.e., the image sample set described in this embodiment) that meets the input requirements of the scanning image processing model. Pre-training the visual feature extractor based on this dataset allows it to better adapt to the specific characteristics of the scanning probe field, thereby enabling more accurate feature extraction.

[0073] In this embodiment, after pre-training the visual feature extractor, the feature extraction module including the visual feature extractor and the Transformer decoder can be trained as a whole. It should be noted that in this embodiment, the Transformer decoder can also be connected to the reconstruction module.

[0074] Specifically, when processing an image, a decoder based on the Transformer architecture can predict the features of the n+1th token from the first n tokens in the input sequence, while also capturing changes in the system state. Specifically applied to the scanning probe image processing field of this embodiment, the Transformer architecture can be pre-trained using large amounts of unlabeled data and efficiently model the spatiotemporal correlations between features.

[0075] The autoregressive model based on the Transformer architecture has a self-attention mechanism. The self-attention mechanism refers to a mechanism that allows the model to automatically pay attention to the relationship between elements at different positions in the sequence when processing sequence data. In the actual processing of the self-attention mechanism, it includes two parts: calculating the attention score and calculating the attention weight. For each element in the sequence (the first feature sequence), it is used as a query (Query) and the similarity is calculated with the key (Key) of all elements in the sequence (including itself) to obtain the attention score. The commonly used similarity calculation method is the dot product, and then the attention score is normalized by the softmax function to obtain the attention weight of each element. The attention weight is weighted and summed with the value (Value) of the corresponding element to obtain a new representation of the current element. This new representation integrates the information of other elements in the sequence and reflects the degree of correlation between the current element and other elements (for example, when applied in the field of scanning probes, such as the spatiotemporal correlation described in this embodiment).

[0076] It is understandable that in this embodiment, due to the presence of the self-attention mechanism, the sample data used when pre-training the decoder can be a large amount of unlabeled data. This is because the self-attention mechanism can automatically focus on the relationship between different elements, without the need for manual labeling for training. Therefore, the decoder using the Transformer architecture can further reduce the manual labeling process, reduce labor costs, and improve training efficiency.

[0077] While the self-attention mechanism captures spatiotemporal correlations, the staggered output characteristics of the Transformer architecture enable the Transformer decoder to predict the next token. In this embodiment, the visual features extracted by the visual feature extractor for each scan segment are used as a token. The Transformer decoder can predict the next visual feature based on all visual features that have appeared in the first feature sequence.

[0078] After obtaining the predicted next visual feature, the predicted visual feature can be input into a reconstruction module (reconstructor). The reconstruction module can reconvert the visual feature into the scanning probe's scanned image space to generate a predicted image. The network architecture of the reconstruction module can generally be a multi-layer neural network or a deconvolutional network, which is not limited in this embodiment.

[0079] It is understood that during model training, the accuracy of the model's generated results can be determined by comparing the differences between the predicted image and the actual image, and the model parameters can be adjusted based on this information. Specifically, in this embodiment, the loss between the predicted image and the actual image data of the next scan segment can be calculated; based on this loss, the parameters of the initial feature extraction module (i.e., the untrained model architecture) are adjusted to train the feature extraction module.

[0080] In some embodiments, the mean square error loss function can be used to Calculate the loss, where L is the loss, To predict the image, is the real image data.

[0081] In summary, the overall training process of the feature extraction module can be as follows Figure 7 As shown. Figure 7 In the example, the tokens output by the visual feature extractor can be aggregated into a feature sequence (with a total of i+1 elements), represented from the 0th element to the i-th element as: {S, h1, …, hi}, where S refers to the "START vector", which is used to mark the starting position 0 of the sequence for recognition by the Transformer decoder.

[0082] ④Task processing module ( Figure 7 (not shown)

[0083] For the parts not described in detail here, please refer to the detailed description in step S130.

[0084] Step S130 , performing preset task processing on the second feature sequence through the task processing module of the scan image processing model to obtain a scan image processing result.

[0085] In this embodiment, the Transformer decoder can be connected to a task processing module. The task processing module is used to handle downstream image processing tasks. The content contained in the task processing module varies depending on the specific processing task involved. For example, downstream image processing tasks may include scanned image classification tasks, specific content search tasks, image quality detection tasks, etc. The following uses some typical examples to explain the application of the scanned image processing model provided by this embodiment in downstream tasks.

[0086] In some embodiments, when a scanned image processing model is applied to an image classification task, the scanned image (assuming it includes n scan segments) obtained from the actual scan can be input into the scanned image processing model. After passing through the visual feature extractor and the Transformer decoder, a second feature vector sequence {Z1,…,Zn} is obtained. This feature sequence is then converted into a one-dimensional vector sequence zavg = AvgPool({Z1,…,Zn}) through a pooling operation. In this example, the task processing module may include an image classification model fhead1 (typically a multi-layer fully connected network). Inputting the one-dimensional vector sequence into the image classification model fhead1 classifies the scanned image and outputs the scanned image category. fhead1 can also be pre-trained using a pre-collected sample dataset. The classification task can be customized by the operator, for example, to determine whether the scanned image is good or bad, but this embodiment is not limited to this. It is understood that during the pre-training process, the backbone network (i.e., the feature extraction module) can be frozen, and only fhead1 can be trained. Alternatively, all model parameters can be trained simultaneously during the feature extraction module training process, but this embodiment is not limited to this.

[0087] In other embodiments, when the scanned image processing model is applied to the probe quality detection task, the portion of the image that has been scanned can be input into the scanned image processing model, and the visual feature extractor can extract the following features: Figure 7 The feature sequence {S, h1, …, hi} is then fed into the Transformer decoder. The Transformer decoder processes h1 through hi to produce a feature vector sequence Zi = fenc(h1, …, hi). Similar to the previous example, Zi can be fed into a pre-trained classification model fhead2 to classify Zi. For example, the classification task might be to determine whether the most recent row is good or bad.

[0088] It can be understood that the image classification task and the probe quality detection task belong to different dimensions. The image classification task belongs to the dimension of the entire image, and its input is a complete scanned image; while the probe quality detection task belongs to the scan segment dimension, and its input is an image being scanned, which may be only a part of the complete scanned image.

[0089] For other tasks such as object detection and segmentation, as an example, a decoder can be used as the backbone network, and a suitable decoding network can be added after the backbone network. For example, for an object detection model, a detection box confidence, coordinate regression, and classification detection head can be added. This example will not be described in detail here.

[0090] This embodiment provides a scanned image processing method. Since the scanned image has the characteristic that its pixel areas are scanned sequentially, the feature extraction module uses the self-attention mechanism of the decoder of the Transformer architecture to extract the correlation between the visual features based on the visual features extracted by the visual feature extractor, so that the second feature sequence extracted by the feature extraction module can fully explore the spatiotemporal correlation in the visual features. When applied to downstream scanned image processing tasks, the accuracy of task processing can be improved.

[0091] The present application provides a scanning image processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the scanning image processing method in the above-mentioned embodiment one.

[0092] Reference below Figure 8 , which shows a schematic structural diagram of a scanning image processing device suitable for implementing the embodiments of the present application. The scanning image processing device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The scanning image processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0093] like Figure 8As shown, the scanning image processing device 200 may include a processing device 210 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 220 or programs loaded from a storage device 230 into a random access memory (RAM) 240. RAM 240 also stores various programs and data required for the operation of the scanning image processing device. Processing device 210, ROM 220, and RAM 240 are interconnected via a bus 250. An input / output (I / O) interface 260 is also connected to the bus. Typically, the following systems may be connected to I / O interface 260: input devices 270, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 280, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 230, such as a magnetic tape, hard disk, etc.; and communication device 290. The communication device 290 can allow the scanning image processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a scanning image processing device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or provided instead.

[0094] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 230, or installed from a ROM 220. When the computer program is executed by the processing device 210, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0095] The scanned image processing device provided in this application, employing the scanned image processing method of the aforementioned embodiment, can solve the technical problem of improving the processing efficiency and accuracy of scanned images. Compared to related technologies, the beneficial effects of the scanned image processing device provided in this application are the same as those of the scanned image processing method provided in the aforementioned embodiment. Other technical features of the scanned image processing device are the same as those disclosed in the aforementioned embodiment and are not further described here.

[0096] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0097] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0098] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the scan image processing method in the above embodiment.

[0099] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0100] The computer-readable storage medium may be included in the scanning image processing device; or may exist independently without being assembled into the scanning image processing device.

[0101] The computer-readable storage medium carries one or more programs that, when executed by the scanning image processing device, enable the scanning image processing device to write computer program code for performing the operations of the present application in one or more programming languages or a combination thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0103] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0104] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned scanned image processing method. This computer-readable storage medium addresses the technical problem of improving the efficiency and accuracy of scanned image processing. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the scanned image processing method provided in the aforementioned embodiments and are not further elaborated here.

[0105] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned scan image processing method when executed by a processor.

[0106] The computer program product provided in this application can solve the technical problem of how to improve the processing efficiency and accuracy of scanned images. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the scanned image processing method provided in the above embodiment, and will not be elaborated here.

[0107] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A scanned image processing method, characterized in that: The scanned image processing method comprises: Inputting a sample scanned image into an initial feature extraction module of a scanned image processing model to obtain a second feature sequence sample output by the initial feature extraction module; the sample scanned image includes a plurality of sample scan segments obtained by sequential scanning, and the initial feature extraction module includes the visual feature extractor and the Transformer decoder; the second feature sequence sample includes sample image features corresponding one-to-one to the plurality of sample scan segments, and the last sample image feature in the second feature sequence sample is a visual feature predicted for the next scan segment of the corresponding sample scan segment; Obtaining a predicted image according to the features of the last sample image; Calculating the loss between the predicted image and the real image data of the next scanning segment; Based on the loss, adjusting the parameters of the initial feature extraction module to obtain a trained feature extraction module; Acquire a scanned image; the scanned image includes a plurality of scan segments obtained by sequential scanning; Extract features from the scanned image using the trained feature extraction module of the scanned image processing model to obtain a second feature sequence; The task processing module of the scanning image processing model performs preset task processing on the second feature sequence to obtain a scanning image processing result.

2. The scanned image processing method according to claim 1, wherein: The calculating the loss between the predicted image and the real image data of the next scanning segment includes: Calculate the loss by expression 1; The expression 1 is: ; Wherein, L is the loss, is the predicted image, is the real image data.

3. The scanned image processing method according to claim 1, wherein: Before acquiring a scanned image, the scanned image processing method further includes: The visual feature extractor is pre-trained using an image sample set related to the task application domain.

4. The scanned image processing method according to claim 3, wherein: The pre-training of the visual feature extractor comprises: Pre-train the visual feature extractor using at least one of an autoencoder model, contrastive learning, and a masked autoencoder.

5. The scanned image processing method according to claim 3, wherein: The image sample set is an unlabeled image sample set.

6. The scanned image processing method according to claim 1, wherein: The task processing module is a probe quality detection module; the task processing module of the scanning image processing model performs preset task processing on the second feature sequence to obtain a scanning image processing result, including: Performing pooling processing on the second feature sequence to obtain a one-dimensional feature vector sequence; The one-dimensional feature vector sequence is input into a classification model in the probe quality detection module, and based on an output of the classification model, a probe quality assessment result corresponding to the scanned image is determined.

7. A scanning image processing device, characterized in that: The scan image processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the scan image processing method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the scan image processing method according to any one of claims 1 to 6 are implemented.

9. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the scanned image processing method according to any one of claims 1 to 6 are implemented.

Images