Experimental record text anomaly classification method and device based on image feature processing

Abstract

Embodiments disclosed by the present application disclose an experimental record text anomaly classification method and device based on image feature processing. A specific embodiment of the method includes: performing data enhancement on an original experimental record text image set in a handwritten experimental text scenario to generate an enhanced experimental record text image group set; performing multi-granularity feature extraction on an experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector, and an inter-frame coarse-granularity vector; generating an experimental record text image feature group according to the intra-frame branch matrix, the inter-frame fine-granularity vector, and the inter-frame coarse-granularity vector; scoring the experimental record text image feature group to generate an experimental record text image feature score group; and performing threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group. The embodiment improves the quality of multi-granularity feature extraction and makes the generated binary abnormal label group more interpretable.

Description

TECHNICAL FIELD

[0001] Embodiments disclosed by the present application relate to the technical field of computer technology, and in particular to an experimental record text anomaly classification method and device based on image feature processing. BACKGROUND

[0002] The experimental record text anomaly classification based on image feature processing is a technology for classifying experimental record text anomalies. At present, when classifying experimental record text anomalies, the commonly used method is to determine whether the experimental record text is abnormal according to a predefined threshold (for example, image contrast, edge sharpness) to achieve anomaly classification, and to rely on manual review to achieve anomaly classification.

[0003] However, when the above method is used, the following technical problems often exist:

[0004] The flexibility of anomaly classification by predefined threshold is poor, which is difficult to adapt to complex scenarios, and relies on manual inspection, which is prone to missed detection, resulting in poor quality of multi-granularity feature extraction, low detection accuracy, and increased probability of detecting experimental record text anomalies. SUMMARY

[0005] The part of the present application is used to introduce the concept in a simple form, which will be described in detail in the specific embodiments part. The part of the present application is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] Some embodiments disclosed by the present application propose an experimental record text anomaly classification method and device based on image feature processing to solve the technical problems mentioned in the background technology part.

[0007] In a first aspect, some embodiments disclosed by the present application provide an experimental record text anomaly classification method based on image feature processing, which comprises: performing data enhancement on a set of original experimental record text images in an experimental text handwriting scene to generate a set of enhanced experimental record text image groups; generating a set of experimental record text image tensors according to the set of enhanced experimental record text image groups; performing multi-granularity feature extraction on the set of experimental record text image tensors to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector and an inter-frame coarse-granularity vector; generating an experimental record text image feature group according to the intra-frame branch matrix, the inter-frame fine-granularity vector and the inter-frame coarse-granularity vector; scoring the experimental record text image feature group to generate an experimental record text image feature score group; and performing threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group, wherein the binary abnormal label group includes labels of different abnormal categories.

[0008] In a second aspect, some embodiments of the present disclosure provide an experimental record text anomaly classification device based on image feature processing. The device comprises: a data enhancement unit configured to perform data enhancement on a set of original experimental record text images in an experimental text handwriting scene to generate a set of enhanced experimental record text image groups; a first generation unit configured to generate a set of experimental record text image tensors according to the set of enhanced experimental record text image groups; an input unit configured to perform multi-granularity feature extraction on the set of experimental record text image tensors to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector and an inter-frame coarse-granularity vector; a second generation unit configured to generate an experimental record text image feature group according to the intra-frame branch matrix, the inter-frame fine-granularity vector and the inter-frame coarse-granularity vector; a scoring unit configured to score the experimental record text image feature group to generate a set of experimental record text image feature scores; and a threshold segmentation unit configured to perform threshold segmentation on the set of experimental record text image feature scores to generate a set of binary abnormality labels, wherein the set of binary abnormality labels comprises labels of different abnormality categories.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method described in any of the implementations of the first aspect.

[0011] The above various embodiments disclosed by the present application have the following beneficial effects: the quality of multi-granularity feature extraction is improved, and the probability of detecting experimental record text abnormalities is balanced by using the experimental record text abnormality classification method based on image feature processing of some embodiments of the present application. Specifically, the reason why the quality of multi-granularity feature extraction is poor and the probability of experimental record text abnormalities increases is that the flexibility of abnormality classification by using a predefined threshold is poor, it is difficult to adapt to complex scenes, and it is prone to missed detection due to reliance on manual work, which leads to poor quality of multi-granularity feature extraction, low detection accuracy, and increased probability of detecting experimental record text abnormalities. Based on this, the experimental record text abnormality classification method based on image feature processing of some embodiments of the present application. First, the original experimental record text image set in the experimental text handwriting scene is subjected to data enhancement to generate an enhanced experimental record text image group set. In this way, the robustness to noise and other interference can be improved through data enhancement. Then, according to the enhanced experimental record text image group set, an experimental record text image tensor set is generated. In this way, the data format can be unified for subsequent processing. After that, the experimental record text image tensor set is subjected to multi-granularity feature extraction to obtain an intra-frame branch matrix, an inter-frame fine-grained vector, and an inter-frame coarse-grained vector. In this way, the intra-frame branch matrix can capture the local details of a single image (such as handwriting distortion and stains), and enhance the sensitivity to static abnormalities. The inter-frame fine-grained vector can analyze the small changes between consecutive frames (such as handwriting coherence), and accurately identify dynamic writing abnormalities. The inter-frame coarse-grained vector can extract sequence-level macro features (such as page layout consistency), and avoid paying too much attention to trivial changes. Therefore, the quality of multi-granularity feature extraction is improved. Next, according to the intra-frame branch matrix, the inter-frame fine-grained vector, and the inter-frame coarse-grained vector, an experimental record text image feature group is generated. In this way, the detection accuracy can be improved. Second, the experimental record text image feature group is scored to generate an experimental record text image feature score group. In this way, the probability of abnormalities can be quantified by comprehensively considering multi-dimensional features, so as to balance the probability of detecting experimental record text abnormalities. Finally, the experimental record text image feature score group is subjected to threshold segmentation to generate a binary abnormality label group, wherein the binary abnormality label group includes labels of different abnormality categories. Therefore, the quality of multi-granularity feature extraction is improved, the probability of detecting experimental record text abnormalities is balanced, and the generated binary abnormality label group is more interpretable. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, aspects, and advantages of the present disclosure will become more apparent with reference to the following detailed description when taken in conjunction with the accompanying drawings wherein: Like reference numbers in the figures indicate like elements. It is to be understood that the drawings are not necessarily to scale.

[0013] Figure 1 is a schematic diagram of one application scenario of the experimental record text anomaly classification method based on image feature processing according to some embodiments disclosed by the present application;

[0014] Figure 2 is a flowchart of the experimental record text anomaly classification method based on image feature processing according to some embodiments disclosed by the present application;

[0015] Figure 3 is a structural schematic diagram of the experimental record text anomaly classification device according to some embodiments disclosed by the present application;

[0016] Figure 4 is a structural schematic diagram of an electronic device suitable for implementing some embodiments disclosed by the present application. DETAILED DESCRIPTION

[0017] The embodiments disclosed by the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the scope of protection of the present application.

[0018] In addition, it should be noted that only the parts related to the present application are shown in the drawings for ease of description. The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0019] It should be noted that the concepts of "first", "second", etc. mentioned in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the adjectives "one" and "multiple" mentioned in the present application are illustrative and not limiting, and those skilled in the art should understand that, unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between the devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of these messages or information.

[0022] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0023] Figure 1is a schematic diagram of one application scenario of the experimental record text abnormal classification method based on image feature processing of some embodiments disclosed by the present application.

[0024] In Figure 1 In the application scenario of, first, the original experimental record text image set in the experimental text handwriting scenario is subjected to data enhancement to generate an enhanced experimental record text image group set. The original experimental record text image set includes: original experimental record text image 1, original experimental record text image 2, …, original experimental record text image m. The enhanced experimental record text image group in the enhanced experimental record text image group set includes: morphed text image 1, morphed text image 2, …, morphed text image n. The morphed text image 1 can be a curved morphed text image. The morphed text image 2 can be a wrinkled morphed text image. The morphed text image n can be a broken morphed text image. Each enhanced experimental record text image in the enhanced experimental record text image group is a morphed experimental record text image of different morphing of the original experimental record text image, and has the same size (from the dotted line, it can be seen that morphing does not change the size). It should be understood that Figure 1 The number of original text images and morphed text images in can have any number according to the needs of implementation.

[0025] With reference to Figure 2 , flow 200 according to some embodiments of the experimental record text abnormal classification method based on image feature processing disclosed by the present application is shown. The experimental record text abnormal classification method based on image feature processing includes the following steps:

[0026] Step 201, the original experimental record text image set in the experimental text handwriting scenario is subjected to data enhancement to generate an enhanced experimental record text image group set.

[0027] In some embodiments, the execution subject (for example, a computing device) of the experimental record text abnormal classification method based on image feature processing can perform data enhancement on the original experimental record text image set in the experimental text handwriting scenario to generate an enhanced experimental record text image group set.

[0028] Here, the above experimental text handwriting scenario can refer to a scenario in which text content needs to be handwritten during the experiment. For example, the above experimental text handwriting scenario can refer to a scenario in which students need to record experimental experience by hand during the experiment. The original experimental record text image in the above original experimental record text image set can refer to an experimental record text image that is not subjected to image preprocessing.

[0029] It should be noted that the above computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. For example, the computing device can be the target terminal described above. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules, such as to provide distributed services, or as a single software or software module. It is not specifically limited here.

[0030] Optionally, the execution subject can perform data augmentation on the original experimental record text image set in the experimental text handwritten scene to generate an enhanced experimental record text image group set by the following steps:

[0031] First, for each original handwritten experimental record image in the original experimental record text image set in the experimental text handwritten scene, the following processing steps are performed:

[0032] First sub-step, pixel normalization is performed on the original experimental record text image to generate a normalized experimental record text image pixel matrix.

[0033] Here, the size of the normalized experimental record text image pixel matrix is the same as that of the original experimental record text image.

[0034] As an example, the execution subject can use the gray() function of the NumPy library in the Python language to perform grayscale processing on the original experimental record text image to generate an experimental record text grayscale image. Then, each pixel of the experimental record text grayscale image is divided by 255 to perform normalization processing to obtain a normalized experimental record text image pixel matrix. The pixel range of each element in the normalized experimental record text image pixel matrix is [0, 1].

[0035] Second sub-step, determine the hyperparameter list from the normalized experimental record text image pixel matrix to generate a hyperparameter list, wherein the hyperparameter list includes: reference points, extrusion strength and extrusion direction.

[0036] Herein, the reference point can refer to a center point of the image. The extrusion strength can refer to a degree of deformation of the image. The extrusion direction can refer to a direction of deformation of the image. The extrusion direction is used to control the orientation of the image deformation. Through deformation in different orientations, multi-granularity feature representation of the image can be achieved, so that the detection accuracy can be improved in a complex scene, thereby reducing the probability of abnormality missed detection. The hyperparameter list is used to control the manner of image transformation. The extrusion strength is measured by the mean square error. For example, the greater the mean square error, the greater the degree of deformation of the image. The mean square error is the average of the squares of the differences between the pixel point values in the normalized experimental record text image pixel matrix and the corresponding pixel point values of the original experimental record text image.

[0037] As an example, the execution subject can randomly set the hyperparameter list for the normalized experimental record text image pixel matrix to generate the hyperparameter list. For example, the extrusion strength can refer to 2.0. For example, the extrusion direction can refer to 0 degrees.

[0038] The third sub-step is to establish an experimental record text blank image pixel matrix with the same size as the normalized experimental record text image pixel matrix.

[0039] Herein, the experimental record text blank image pixel matrix can refer to a matrix with all elements being 0. The experimental record text blank image pixel matrix is used to store the transformed pixel values.

[0040] As an example, the execution subject can initialize an empty data structure to establish an experimental record text blank image pixel matrix with the same size as the normalized experimental record text image pixel matrix.

[0041] The fourth sub-step is to generate an experimental record text blank image pixel group for each image pixel point in the experimental record text blank image pixel matrix according to the image pixel point, the reference point and the extrusion strength.

[0042] Optionally, the execution subject can generate an experimental record text blank image pixel group according to the image pixel point, the reference point and the extrusion strength by the following steps:

[0043] Sub-step one is to determine the distance between the image pixel point and the reference point to obtain a pixel reference distance.

[0044] Herein, the image pixel point can refer to a pixel point in the blank image being processed. The pixel reference distance can refer to the Euclidean distance between the image pixel point and the reference point.

[0045] Sub-step two is to extrude the pixel reference distance according to the extrusion strength to generate an extruded pixel reference distance.

[0046] As an example, the execution subject can perform nonlinear compression on the pixel reference distance according to the extrusion strength by using a compression function, to generate an extruded pixel reference distance. For example, the pixel reference distance is 64.03. The extruded pixel reference distance is 32.0. The compression function can be d1 = d / (1+kd), where d1 represents the extruded pixel reference distance, d represents the pixel reference distance, and k represents the extrusion strength.

[0047] Sub-step three, determining a pixel reference direction angle between the image pixel point and the reference point.

[0048] Here, the pixel reference direction angle can refer to the angle from the reference point to the current pixel point.

[0049] As an example, the execution subject can determine the pixel reference direction angle between the image pixel point and the reference point by using a two-parameter arctangent function arctan2().

[0050] Sub-step four, determining the pixel point coordinates in the normalized experimental record text image pixel matrix as the normalized experimental record text image pixel point coordinates according to the pixel reference direction angle and the extruded pixel reference distance.

[0051] As an example, the execution subject can perform polar to rectangular coordinate conversion on the pixel reference direction angle and the extruded pixel reference distance, and the pixel point coordinates of the image pixel point, to obtain the pixel point coordinates in the normalized experimental record text image pixel matrix as the normalized experimental record text image pixel point coordinates. For example, the pixel point coordinates of the image pixel point are (ox, oy), where o represents the reference point, and x and y represent the horizontal and vertical coordinates respectively. ox represents the horizontal coordinate with the reference point as the origin. oy represents the vertical coordinate with the reference point as the origin. The pixel reference direction angle is represented by α, the extruded pixel reference distance is represented by d1, and the normalized experimental record text image pixel point coordinates are represented by (px, py), where px and py represent the horizontal and vertical coordinates of the pixel point coordinates in the normalized experimental record text image pixel matrix respectively. That is, px = ox + d1 × cos(α), py = oy + d1 × sin(α).

[0052] Sub-step five, performing gray value extraction on the normalized experimental record text image pixel point coordinates, and assigning the extracted gray value to the image pixel point to obtain the experimental record text blank image pixel point.

[0053] As an example, the execution subject can use bilinear interpolation to extract the gray value of the pixel point coordinate of the normalized experimental record text image, and store the extracted gray value to the image pixel point to obtain the experimental record text blank image pixel point. For example, four adjacent pixel points of the normalized experimental record text image pixel point coordinate are determined as adjacent pixel point 1, adjacent pixel point 2, adjacent pixel point 3, and adjacent pixel point 4. Then, the adjacent pixel point 1 and the adjacent pixel point 2 are interpolated in the horizontal direction to obtain the interpolation pixel point 1. Next, the adjacent pixel point 3 and the adjacent pixel point 4 are interpolated in the vertical direction to obtain the interpolation pixel point 2. Finally, the interpolation pixel point 1 and the interpolation pixel point 2 are interpolated to obtain the experimental record text blank image pixel point.

[0054] In a fifth sub-step, the obtained each experimental record text blank image pixel point group is determined as an extrusion deformation experimental record text image group.

[0055] Here, the experimental record text blank image pixel point group can refer to an entire deformed image. For example, the experimental record text blank image pixel point group can refer to a 200x200 deformed image.

[0056] In a second step, interference superposition is performed on the obtained extrusion deformation experimental record text image group set to generate an interference experimental record text image set as an enhanced experimental record text image group set.

[0057] As an example, the execution subject can add Gaussian noise to the obtained extrusion deformation experimental record text image group set to generate an added extrusion deformation experimental record text image group set. Then, Gaussian blur is used to blur the added extrusion deformation experimental record text image group set to generate a blurred extrusion deformation experimental record text image group set as an enhanced experimental record text image group set.

[0058] In step 202, an experimental record text image tensor set is generated according to the enhanced experimental record text image group set.

[0059] In some embodiments, the execution subject can generate an experimental record text image tensor set according to the enhanced experimental record text image group set.

[0060] Here, the experimental record text image tensor in the experimental record text image tensor set can refer to an experimental record text image tensor formed by splicing each enhanced experimental record text image in the enhanced experimental record text image group set according to the channel.

[0061] Optionally, the execution subject can generate the experimental record text image tensor set according to the enhanced experimental record text image group set by the following steps:

[0062] Firstly, for each enhanced experimental record text image group in the enhanced experimental record text image group set, the following processing steps are performed:

[0063] Firstly, the preset number of permutations is performed on each enhanced experimental record text image in the enhanced experimental record text image group to generate an experimental record text image permutation group.

[0064] Here, the preset number can refer to a preset number. For example, the preset number can be 4.

[0065] As an example, the execution subject can arrange each enhanced experimental record text image in the enhanced experimental record text image group according to the preset number to generate an experimental record text image permutation group. For example, 20 enhanced experimental record text images are randomly selected from the enhanced experimental record text image group without replacement, and the process is repeated 4 times.

[0066] Secondly, the position vector determination is performed on the experimental record text image permutation group to generate an experimental record text image permutation vector group.

[0067] Here, the experimental record text image permutation vector in the experimental record text image permutation vector group can refer to an enhanced experimental record text image containing a position index.

[0068] As an example, the execution subject can randomly determine the index of the experimental record text image in the experimental record text image permutation group to generate an experimental record text image index group. The experimental record text image in the experimental record text image index group can be an experimental record text image with an index identifier. The range of the index identifier can be [0, 20]. Then, the experimental record text image index group is rearranged to generate an experimental record text image permutation index group as the experimental record text image permutation vector group.

[0069] Thirdly, each experimental record text image permutation vector in the experimental record text image permutation vector group is spliced to generate an experimental record text image tensor.

[0070] Secondly, each experimental record text image tensor obtained is determined as an experimental record text image tensor set.

[0071] In step 203, multi-granularity feature extraction is performed on the experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-grained vector and an inter-frame coarse-grained vector.

[0072] In some embodiments, the execution subject can perform multi-granularity feature extraction on the experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector, and an inter-frame coarse-granularity vector.

[0073] Here, the intra-frame branch matrix can represent static appearance features of at least one experimental record text image. The static appearance features can refer to visual elements in a single experimental record text image that do not change over time and can be observed by the naked eye. For example, the static appearance features can refer to stroke topology. The stroke topology includes at least one of the following: horizontal, vertical, hook, and dot. The inter-frame fine-granularity vector represents micro-jitter features of at least one experimental record text image. The micro-jitter features can refer to a 1-pixel drift up and down in a single experimental record text image. The inter-frame coarse-granularity vector can represent large deformation features of at least one experimental record text image. The large deformation features can refer to bending features or wrinkle features.

[0074] In the process of solving the problems mentioned in the background by using the technical solutions, the following problems often occur:

[0075] When performing multi-granularity feature extraction on the experimental record text image tensor set, local subtle abnormalities and internal spatial structures of features can be ignored, resulting in a large error in the scoring result, poor stability of abnormality detection, and inability to quantitatively identify abnormal regions.

[0076] In the face of the above technical problems, the inventors decided to use the following solutions:

[0077] Alternatively, the execution subject can perform multi-granularity feature extraction on the experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector, and an inter-frame coarse-granularity vector by the following steps:

[0078] First, quantitatively segmenting the experimental record text image tensor set to generate an experimental record text image cube set.

[0079] Here, the experimental record text image cube in the experimental record text image cube set can be a small three-dimensional data block segmented from the experimental record text image tensor set.

[0080] As an example, the execution subject can use a sliding window method to quantitatively segment the experimental record text image tensor set to generate an experimental record text image cube set. The sliding window method can be to slide in the width, height, and time dimensions of the image sequence with a fixed step (e.g., 4px) with a width x height x time depth of 8px x 8px x 5 frames.

[0081] Secondly, a preset number of experimental record text image cubes in the experimental record text image cube set are compressed and transformed to generate experimental record text image low-frequency vectors, and an experimental record text image low-frequency vector sequence is obtained.

[0082] As an example, the execution subject can compress and transform a preset number of experimental record text image cubes in the experimental record text image cube set by three-dimensional discrete cosine transform to generate experimental record text image low-frequency vectors, and an experimental record text image low-frequency vector sequence is obtained.

[0083] Thirdly, each frame of experimental record text image low-frequency vector in the experimental record text image low-frequency vector sequence is multi-granularity convolved to generate an intra-frame branch vector, and an intra-frame branch vector set is obtained.

[0084] As an example, the execution subject can use a plurality of one-dimensional convolution kernels with different kernel sizes to multi-granularity convolve each frame of experimental record text image low-frequency vector in the experimental record text image low-frequency vector sequence to generate an intra-frame branch vector, and an intra-frame branch vector set is obtained. Wherein, the plurality of one-dimensional convolution kernels with different kernel sizes can be a plurality of convolution kernels with kernel sizes of 3, 5, and 7.

[0085] Fourthly, each intra-frame branch vector in the intra-frame branch vector set is spliced to obtain an intra-frame branch matrix.

[0086] As an example, the execution subject can stack each intra-frame branch vector in the intra-frame branch vector set in the row direction to obtain an intra-frame branch matrix. For example, assuming that each intra-frame branch vector is a row vector with a length of L, after splicing, a matrix with a shape of N×L is obtained for a sequence containing N frames.

[0087] Fifthly, a preset channel convolution is performed on the experimental record text image low-frequency vector sequence to generate a convolution experimental record text image low-frequency vector sequence.

[0088] Here, the preset channel can refer to a preset channel dimension.

[0089] As an example, the execution subject can perform convolution on the experimental record text image low-frequency vector sequence in the preset channel to generate a convolution experimental record text image low-frequency vector sequence.

[0090] Sixthly, peak extraction is performed on the convolution experimental record text image low-frequency vector sequence to generate an experimental record text image low-frequency vector peak sequence.

[0091] As an example, the execution subject can identify points in the convolutional experimental record text image low-frequency vector sequence that are greater than both their adjacent points (local maxima) or smaller than both their adjacent points (local minima) as a peak point set. Then, the peak point set is arranged in ascending order to generate an experimental record text image low-frequency vector peak sequence.

[0092] Step 7: compress the pooled experimental record text image low-frequency vector peak sequence to generate an inter-frame fine-grained vector.

[0093] As an example, the execution subject can compress the pooled experimental record text image low-frequency vector peak sequence through an encoder structure to generate an inter-frame fine-grained vector. The encoder structure can refer to a fully connected network or a recurrent neural network.

[0094] Step 8: average pool the experimental record text image low-frequency vector sequence to generate a pooled experimental record text image low-frequency vector.

[0095] As an example, the execution subject can perform an arithmetic mean operation on the experimental record text image low-frequency vector sequence to generate a pooled experimental record text image low-frequency vector.

[0096] Step 9: multi-scale convolution on the pooled experimental record text image low-frequency vector to generate a multi-scale experimental record text image low-frequency vector.

[0097] As an example, the execution subject can use multiple one-dimensional convolution kernels of different lengths to perform multi-scale convolution on the pooled experimental record text image low-frequency vector to generate a multi-scale experimental record text image low-frequency vector.

[0098] Step 10: maximum value extraction on the multi-scale experimental record text image low-frequency vector to generate a vector maximum value set.

[0099] As an example, the execution subject can determine the maximum value in the multi-scale experimental record text image low-frequency vector as the vector maximum value set.

[0100] Step 11: concatenation of the vector maximum value set to generate an inter-frame coarse-grained vector.

[0101] As an example, the execution subject can concatenate the vector maximum value set in descending order to generate an inter-frame coarse-grained vector.

[0102] The related content in the first step to the tenth step above is disclosed as an invention point of the present application, which solves the technical problem of "poor stability of anomaly detection and inability to quantitatively identify abnormal regions". The factors that cause poor stability of anomaly detection and inability to quantitatively identify abnormal regions are often as follows: when performing multi-granularity feature extraction on the experimental record text image tensor set, local subtle abnormalities and internal spatial structures of features may be ignored, resulting in a large error in the score result, poor stability of anomaly detection, and inability to quantitatively identify abnormal regions. If the above factors are solved, the stability of anomaly detection and the effect of quantitatively identifying abnormal regions can be improved. In order to achieve this effect, first, the experimental record text image tensor set is quantized and divided to generate an experimental record text image cube set. Therefore, the continuous image sequence is structured and cut into smaller three-dimensional data blocks, which can reduce the complexity of subsequent processing. Then, a preset number of experimental record text image cubes in the experimental record text image cube set are compressed and transformed to generate an experimental record text image low-frequency vector, obtaining an experimental record text image low-frequency vector sequence. In this way, the low-frequency components of the cube can be extracted, thereby realizing data dimension reduction. Next, each experimental record text image low-frequency vector in the experimental record text image low-frequency vector sequence is subjected to multi-granularity convolution to generate an intra-frame branch vector, obtaining an intra-frame branch vector set. Therefore, different scale convolution kernels can be used to scan a single-frame low-frequency vector, which can capture multi-level spatial features from subtle strokes to larger text regions, thereby comprehensively describing the static content of each frame of image. Then, each intra-frame branch vector in the intra-frame branch vector set is spliced to obtain an intra-frame branch matrix. A preset channel convolution is performed on the experimental record text image low-frequency vector sequence to generate a convolution experimental record text image low-frequency vector sequence. Peak extraction is performed on the convolution experimental record text image low-frequency vector sequence to generate an experimental record text image low-frequency vector peak sequence. Therefore, the most significant moments in the time sequence can be highlighted, for example, the beginning of the stroke, the end of the stroke or the moment of rapid movement, thereby realizing fine-grained focusing on time dynamics. Subsequently, the pooled experimental record text image low-frequency vector peak sequence is compressed to generate an inter-frame fine-grained vector. The experimental record text image low-frequency vector sequence is subjected to average pooling to generate a pooled experimental record text image low-frequency vector. In this way, the overall trend of the sequence can be retained while short-term and subtle fluctuations are ignored. Secondly, multi-scale convolution is performed on the pooled experimental record text image low-frequency vector to generate a multi-scale experimental record text image low-frequency vector. Maximum value extraction is performed on the multi-scale experimental record text image low-frequency vector to generate a vector maximum value set. In this way, the most prominent global features at different time scales can be captured. The vector maximum value set is spliced to generate an inter-frame coarse-grained vector.Therefore, by extracting the intra branch matrix, the inter fine-grained vector and the inter coarse-grained vector, the stability of the anomaly detection is improved, and the identified abnormal region can be quantified.

[0103] In step 204, the experimental record text image feature group is generated according to the intra branch matrix, the inter fine-grained vector and the inter coarse-grained vector.

[0104] In some embodiments, the execution subject can generate the experimental record text image feature group according to the intra branch matrix, the inter fine-grained vector and the inter coarse-grained vector.

[0105] Here, the experimental record text image feature group can refer to 20 enhanced experimental record text image features of 64x64.

[0106] As an example, the execution subject can splice the intra branch matrix, the inter fine-grained vector and the inter coarse-grained vector to obtain the experimental record text image feature group.

[0107] In step 205, the experimental record text image feature group is scored to generate an experimental record text image feature score group.

[0108] In some embodiments, the execution subject can score the experimental record text image feature group to generate an experimental record text image feature score group.

[0109] Here, the experimental record text image feature score group can represent the deformation degree of the experimental record text image feature. The experimental record text image feature score in the experimental record text image feature score group ranges from 0 to 1. The closer the experimental record text image feature score is to 0, the greater the deformation. The closer the experimental record text image feature score is to 1, the smaller the deformation.

[0110] Optionally, the execution subject can score the experimental record text image feature group to generate an experimental record text image feature score group by the following steps:

[0111] First, the experimental record text image feature group is pre-sized to generate an experimental record text image feature block.

[0112] Here, the pre-set size can refer to a pre-set size. For example, the pre-set size can be 4x4x4.

[0113] As an example, the execution subject can perform preset size segmentation on the experimental record text image feature group by using the unfold function and the reshape function to generate experimental record text image feature blocks. For example, a 64x64 experimental record text image feature group is segmented into 256 4x4x4 experimental record text image feature blocks.

[0114] Secondly, each experimental record text image feature in the experimental record text image feature block is compressed and transformed to generate an experimental record text image low-frequency coefficient, thereby obtaining an experimental record text image low-frequency coefficient sequence.

[0115] Here, the experimental record text image low-frequency coefficient sequence can refer to a sequence of low-frequency coefficients representing high-energy information arranged from small to large after the experimental record text image features are compressed and transformed. The high-energy information can refer to bending information or wrinkle information.

[0116] As an example, the execution subject can perform compression and transformation on each experimental record text image feature in the experimental record text image feature block by using a three-dimensional discrete cosine transform (3-D DCT) to generate an experimental record text image low-frequency coefficient, thereby obtaining an experimental record text image low-frequency coefficient sequence.

[0117] Thirdly, a preset number of experimental record text image low-frequency coefficients in the experimental record text image low-frequency coefficient sequence are matrix-pasted to generate a low-frequency coefficient matrix.

[0118] Here, the preset number can refer to the number of low-frequency coefficients that are preset. For example, the preset number can refer to 8.

[0119] As an example, the execution subject can arrange the first preset number of experimental record text image low-frequency coefficients in the experimental record text image low-frequency coefficient sequence in a matrix to generate a low-frequency coefficient matrix. For example, 8 experimental record text image low-frequency coefficients are arranged in a column to generate a 256 row x 8 column low-frequency coefficient matrix.

[0120] Fourthly, row similarity determination is performed on the low-frequency coefficient matrix to generate a row similarity set.

[0121] As an example, the execution subject can perform row cosine similarity determination on the low-frequency coefficient matrix to generate a row similarity set.

[0122] Fifthly, matrix normalization is performed on the row similarity set to generate a normalized matrix.

[0123] As an example, the execution subject can arrange each row similarity in the row similarity set row by row, and divide each element in the arranged similarity matrix by the sum of elements in the arranged similarity matrix to obtain a normalized matrix.

[0124] In the sixth step, the normalized matrix is subjected to spectral clustering to generate a spectral clustering label set.

[0125] Here, the spectral clustering labels in the spectral clustering label set can represent spectral clustering labels with labels of "0" and labels of "1" after classification. The label of "0" can represent normal. The label of "1" can represent abnormal. For example, the label of "1" can represent the position of the fracture.

[0126] As an example, the execution subject can perform spectral clustering on the normalized matrix by k-means to generate a spectral clustering label set.

[0127] In the seventh step, a consistency score set is generated according to the spectral clustering label set.

[0128] As an example, the execution subject can determine the average value of the similarity of the same labels in the spectral clustering label set to generate a close average value. Then, the average value of the similarity of different labels in the spectral clustering label set is determined to generate a distant average value. Then, the close average value is subtracted from the distant average value and divided by the sum of the close average value and the distant average value to obtain a block-level consistency score, thereby obtaining a consistency score set. The range of the consistency score set is [-1, 1].

[0129] In the eighth step, the consistency score set is mapped to a low-frequency coefficient trust weight to obtain a block-level anomaly weight map.

[0130] Here, the block-level anomaly weight map includes bright white areas and dark areas, wherein the bright white areas represent normal scores, and the dark areas represent abnormal scores.

[0131] As an example, the execution subject can compress each consistency score in the consistency score set into a trust weight set by an S-shaped function. The range of the trust weight set is [0, 1].

[0132] In the ninth step, the block-level anomaly weight map is filled into the space corresponding to the experimental record text image feature group to obtain an anomaly confidence heat map.

[0133] Here, the abnormal confidence heat map can refer to a visualized image obtained by mapping the block-level abnormal weight map to the space corresponding to the feature set of the experimental record text image. The mapping refers to mapping from a low-dimensional space to a high-dimensional space. For example, the block-level abnormal weight map with a resolution of 16x12 is converted to an abnormal confidence heat map with a resolution of 1024x768 by interpolation.

[0134] As an example, the execution subject can first determine the spatial position coordinates corresponding to each feature block in the space corresponding to the feature set of the experimental record text image. Then, the block-level abnormal weight map is filled into the spatial position coordinates to obtain a filled heat map. Finally, the size of the filled heat map is enlarged by an upsampling algorithm (such as bilinear interpolation) to obtain the abnormal confidence heat map. The size enlargement can be to enlarge the size of the filled heat map to the size of the space corresponding to the feature set of the experimental record text image.

[0135] In the tenth step, the abnormal confidence heat map is globally averaged to generate the experimental record text image feature score set.

[0136] As an example, the execution subject can divide the weight value of all pixel points in the abnormal confidence heat map by the total number of all pixel points in the abnormal confidence heat map to obtain the experimental record text image feature score set.

[0137] The above-mentioned related contents in the first step to the tenth step are disclosed as an invention point of the present application, which solves the technical problem of "unable to effectively evaluate the severity level of different anomalies". The factors that lead to the inability to accurately locate the area where the anomaly occurs and effectively evaluate the severity level of different anomalies are often as follows: when scoring the experimental record text image feature group, the output is only a single scalar, which cannot accurately locate the area where the anomaly occurs, cannot effectively evaluate the severity level of different anomalies, cannot distinguish between minor flaws and serious errors, and leads to poor interpretability of the detection results. If the above factors are solved, the effect of visualizing the location of the area corresponding to the severity level of different anomalies can be achieved. In order to achieve this effect, first, the experimental record text image feature group is pre-set size segmented to generate experimental record text image feature blocks. Therefore, the global feature map is decomposed into local feature blocks, and the granularity of anomaly detection is refined from "the whole image" to "local area", which provides convenience for subsequent positioning of local anomalies. Each experimental record text image feature in the experimental record text image feature block is compressed and transformed to generate experimental record text image low-frequency coefficients, and an experimental record text image low-frequency coefficient sequence is obtained. In this way, the interference of high-frequency noise can be effectively filtered. A pre-set number of experimental record text image low-frequency coefficients in the experimental record text image low-frequency coefficient sequence are matrix spliced to generate a low-frequency coefficient matrix. The row similarity of the low-frequency coefficient matrix is determined to generate a row similarity set. In this way, by calculating the similarity between each two rows in the matrix, the similarity of the features of different regions of the image is quantified, and the region with low similarity implies potential anomalies. The row similarity set is matrix normalized to generate a normalized matrix. The normalized matrix is spectrally clustered to generate a spectral clustering label set. According to the spectral clustering label set, a consistency score set is generated. In this way, the area where the anomaly occurs can be located by spectral clustering, and the severity level of different anomalies can be effectively evaluated by the consistency score set. The consistency score set is mapped to the low-frequency coefficient credible weight to obtain a block-level anomaly weight map. Therefore, it can be clearly indicated which local area in the image may have a problem. The block-level anomaly weight map is filled into the space corresponding to the experimental record text image feature group to obtain an anomaly confidence heat map. Therefore, the location of the area corresponding to the severity level of different anomalies can be visualized. The anomaly confidence heat map is globally averaged to generate an experimental record text image feature score group. Therefore, whether the image has an anomaly can be determined by the anomaly confidence heat map. The area where the anomaly occurs is located by the weight map, thereby effectively evaluating the severity level of different anomalies, thereby distinguishing between minor flaws and serious errors, and improving the interpretability of the detection results.

[0138] In step 206, the experimental record text image feature score group is threshold segmented to generate a binary anomaly label group.

[0139] In some embodiments, the execution subject can perform threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group, wherein the binary abnormal label group includes labels of different abnormal categories.

[0140] Optionally, the execution subject can perform threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group by the following steps:

[0141] Firstly, average determination is performed on the experimental record text image feature score group to generate an experimental record text image feature average score.

[0142] For example, the execution subject can perform arithmetic mean determination on the experimental record text image feature score group to generate an experimental record text image feature average score.

[0143] Secondly, comparison is performed between the experimental record text image feature score group and the experimental record text image feature average score to obtain a comparison result group.

[0144] For example, the execution subject can perform subtraction operation between each experimental record text image feature score in the experimental record text image feature score group and the experimental record text image feature average score to generate an operation result, and obtain an operation result group as the comparison result group.

[0145] Thirdly, preset first threshold label assignment is performed on at least one experimental record text image feature score in the comparison result group that meets a preset normal false positive rate and is less than the experimental record text image feature average score to generate a first threshold assignment label group.

[0146] Here, the preset threshold label can be "0". The assignment can be a mark. The first threshold assignment label group represents a threshold assignment label group with normal scores.

[0147] The preset normal false positive rate can be a pre-set normal false positive rate. For example, the preset normal false positive rate can be 5%.

[0148] For example, the execution subject can perform 95th percentile determination on at least one experimental record text image feature score that is less than the experimental record text image feature average score to obtain a sub-threshold, wherein 95% of the scores are lower than the sub-threshold. At least one experimental record text image feature score that is less than the sub-threshold in the experimental record text image feature average score is marked as "0".

[0149] In the fourth step, a preset first threshold label is assigned to at least one experimental record text image feature score in the comparison result group that does not satisfy the preset normal false positive rate and is less than the average experimental record text image feature score, to generate a second threshold label group.

[0150] Here, the second threshold label group can represent a threshold label group of slight abnormalities.

[0151] In the fifth step, a preset second threshold label is assigned to at least one experimental record text image feature score in the comparison result group that is greater than the average experimental record text image feature score, to generate a third threshold label group.

[0152] Here, the preset second threshold label can be "1". The third threshold label group can represent a threshold label group of abnormalities.

[0153] In the sixth step, the first threshold label group, the second threshold label group, and the third threshold label group are determined as a binary abnormality label group.

[0154] As an example, the first threshold label group, the second threshold label group, and the third threshold label group can be combined into a binary abnormality label group by the execution subject. Therefore, the preset normal false positive rate, the preset first threshold label, and the preset second threshold label can adapt to changes in experimental record text image feature scores, avoiding the rigidity of a single fixed threshold, and increasing the credibility of the binary abnormality label group.

[0155] Further referring to Figure 3 , as an implementation of the methods shown in the above figures, the disclosure provides some embodiments of an experimental record text abnormality classification device based on image feature processing, which corresponds to the method embodiments shown in Figure 2 The experimental record text abnormality classification device based on image feature processing can be applied in various electronic devices.

[0156] As Figure 3As shown, the experimental record text anomaly classification device 300 based on image feature processing of some embodiments includes a data enhancement unit 301, a first generation unit 302, an input unit 303, a second generation unit 304, a scoring unit 305, and a threshold segmentation unit 306. Among them, the data enhancement unit 301 is configured to perform data enhancement on the original experimental record text image set in the experimental text handwriting scene to generate an enhanced experimental record text image group set; the first generation unit 302 is configured to generate an experimental record text image tensor set according to the above-mentioned enhanced experimental record text image group set; the input unit 303 is configured to perform multi-granularity feature extraction on the above-mentioned experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-grained vector and an inter-frame coarse-grained vector; the second generation unit 304 is configured to generate an experimental record text image feature group according to the above-mentioned intra-frame branch matrix, the above-mentioned inter-frame fine-grained vector and the above-mentioned inter-frame coarse-grained vector; the scoring unit 305 is configured to score the above-mentioned experimental record text image feature group to generate an experimental record text image feature score group; the threshold segmentation unit 306 is configured to perform threshold segmentation on the above-mentioned experimental record text image feature score group to generate a binary abnormal label group, wherein the above-mentioned binary abnormal label group includes labels of different abnormal categories.

[0157] It can be understood that the units described in the experimental record text anomaly classification device 300 based on image feature processing correspond to the respective steps in the method described above. Figure 2 Therefore, the operations, features and beneficial effects described above for the method also apply to the experimental record text anomaly classification device 300 based on image feature processing and the units contained therein, which will not be repeated here.

[0158] Reference is made below to Figure 4 which shows a structural schematic diagram of an electronic device (for example, a computing device 401) suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure. As Figure 4As shown, the computer device includes a processor 402, a memory and a network interface 403 connected through a system bus 406, wherein the memory can include a non-volatile storage medium 404 and an internal memory 405. The non-volatile storage medium 404 can store an operating system 4041 and a computer program 4042. The computer program 4042 includes program instructions which, when executed, can cause the processor 402 to perform any of the above methods. The processor 402 is configured to provide computing and control capabilities to support the operation of the entire computer device. The internal memory 405 provides an environment for the execution of the computer program 4042 in the non-volatile storage medium 404, which, when executed by the processor 402, can cause the processor 402 to perform any of the above methods. The network interface 403 is configured to perform network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that, Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the disclosed scheme, and does not constitute a limitation on the computer device to which the disclosed scheme is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0159] It should be understood that the processor 402 can be a central processing unit (CPU), and the processor 402 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0160] In one embodiment, the processor 402 is configured to run the computer program 4042 stored in the memory to implement the following steps: performing data augmentation on a set of original experimental record text images in an experimental text handwritten scenario to generate an augmented experimental record text image group set; generating a set of experimental record text image tensors according to the augmented experimental record text image group set; performing multi-granularity feature extraction on the set of experimental record text image tensors to obtain an intra-frame branch matrix, an inter-frame fine-grained vector, and an inter-frame coarse-grained vector; generating an experimental record text image feature group according to the intra-frame branch matrix, the inter-frame fine-grained vector, and the inter-frame coarse-grained vector; scoring the experimental record text image feature group to generate an experimental record text image feature score group; and performing threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group, wherein the binary abnormal label group includes labels of different abnormal categories.

[0161] The embodiments of the method disclosed in the present application also provide a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program includes program instructions. The program instructions are executed to implement the method. The method can refer to the embodiments of the method disclosed in the present application.

[0162] The computer readable storage medium can be an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0163] It should be noted that, in the present document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0164] The above description is merely some of the preferred embodiments of the present disclosure and a description of the technical principles of the present disclosure. It should be understood by those skilled in the art that the inventive scope of the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the above inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) form the technical solutions.

Claims

1. An experimental record text anomaly classification method based on image feature processing, characterized in that, The method comprises the following steps: data augmentation is performed on a set of original experimental record text images in a handwritten experimental text scenario to generate a set of augmented experimental record text image groups; a set of experimental record text image tensors is generated according to the set of augmented experimental record text image groups; multi-granularity feature extraction is performed on the set of experimental record text image tensors to obtain an intra-frame branch matrix, an inter-frame fine-grained vector and an inter-frame coarse-grained vector; experimental record text image features are generated according to the intra-frame branch matrix, the inter-frame fine-grained vector and the inter-frame coarse-grained vector; scoring is performed on the set of experimental record text image features to generate a set of experimental record text image feature scores; threshold segmentation is performed on the set of experimental record text image feature scores to generate a set of binary abnormal labels, wherein the set of binary abnormal labels includes labels of different abnormal categories. The method comprises the following steps: for each augmented experimental record text image group in the set of augmented experimental record text image groups, the following processing steps are performed: a preset number of permutations are performed on each augmented experimental record text image in the augmented experimental record text image group to generate a set of experimental record text image permutation groups; position vector determination is performed on the set of experimental record text image permutation groups to generate a set of experimental record text image permutation vector groups; each experimental record text image permutation vector in the set of experimental record text image permutation vector groups is spliced to generate an experimental record text image tensor; each experimental record text image tensor obtained is determined as a set of experimental record text image tensors.

2. The method of claim 1, wherein, The method comprises the following steps: for each original experimental record text image in the set of original experimental record text images in a handwritten experimental text scenario, the following processing steps are performed: pixel normalization is performed on the original experimental record text image to generate a normalized experimental record text image pixel matrix; a hyperparameter list is determined from the normalized experimental record text image pixel matrix to generate a hyperparameter list, wherein the hyperparameter list includes a reference point, a squeezing intensity and a squeezing direction; an experimental record text blank image pixel matrix with the same size as the normalized experimental record text image pixel matrix is established; for each image pixel point in the experimental record text blank image pixel matrix, an experimental record text blank image pixel point group is generated according to the image pixel point, the reference point and the squeezing intensity; each experimental record text blank image pixel point group obtained is determined as a set of squeezed and deformed experimental record text images; interference is superimposed on the set of squeezed and deformed experimental record text images to generate a set of interference experimental record text images as a set of augmented experimental record text image groups.

3. The method of claim 2, wherein, The method comprises the following steps: Determine the distance between the image pixel point and the reference point to obtain a pixel reference distance; According to the extrusion strength, the pixel reference distance is extruded to generate an extruded pixel reference distance; According to the image pixel point and the reference point, the pixel reference direction angle is determined; According to the pixel reference direction angle and the extruded pixel reference distance, the pixel point coordinates corresponding to the normalized experimental record text image pixel matrix are determined as the normalized experimental record text image pixel coordinates; The gray value of the normalized experimental record text image pixel coordinates is extracted, and the extracted gray value is assigned to the image pixel point to obtain the experimental record text blank image pixel point.

4. The method of claim 1, wherein, The experimental record text image feature score group is threshold segmented to generate a binary abnormal label group, including: The experimental record text image feature score group is averaged to determine the experimental record text image feature average score; The experimental record text image feature score group is compared with the experimental record text image feature average score to obtain a comparison result group; At least one experimental record text image feature score in the comparison result group that meets the preset normal false alarm rate and is less than the experimental record text image feature average score is assigned a preset first threshold label to generate a first threshold assignment label group; At least one experimental record text image feature score in the comparison result group that does not meet the preset normal false alarm rate and is less than the experimental record text image feature average score is assigned a preset second threshold label to generate a second threshold assignment label group; At least one experimental record text image feature score in the comparison result group that is greater than the experimental record text image feature average score is assigned a preset third threshold label to generate a third threshold assignment label group; The first threshold assignment label group, the second threshold assignment label group and the third threshold assignment label group are determined as the binary abnormal label group.

5. An experimental record text abnormality classification device based on image feature processing, characterized by, Including: The data enhancement unit is configured to perform data enhancement on the original experimental record text image set in the experimental text handwriting scene to generate an enhanced experimental record text image group set; The first generation unit is configured to generate an experimental record text image tensor set according to the enhanced experimental record text image group set; wherein the generation of the experimental record text image tensor set according to the enhanced experimental record text image group set includes: For each enhanced experimental record text image group in the enhanced experimental record text image group set, the following processing steps are performed: The experimental record text image arrangement group is subjected to position vector determination to generate an experimental record text image arrangement vector group; The experimental record text image arrangement vectors in the experimental record text image arrangement vector group are spliced to generate an experimental record text image tensor; The obtained experimental record text image tensors are determined as the experimental record text image tensor set; ​ The input unit is configured to perform multi-granularity feature extraction on the experimental record text image tensor set to obtain an intra-frame branch matrix, an inter-frame fine-granularity vector, and an inter-frame coarse-granularity vector. The second generation unit is configured to generate an experimental record text image feature group according to the intra-frame branch matrix, the inter-frame fine-granularity vector, and the inter-frame coarse-granularity vector. The scoring unit is configured to score the experimental record text image feature group to generate an experimental record text image feature score group. The threshold segmentation unit is configured to perform threshold segmentation on the experimental record text image feature score group to generate a binary abnormal label group, wherein the binary abnormal label group includes labels of different abnormal categories.

6. An electronic device, comprising: comprise: one or more processors; a memory device having stored thereon one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium characterized by a computer program is stored thereon, wherein the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images