Method and device for identifying scale and scale information combination features of electron microscope images

Abstract

Embodiments of the present invention relate to a method and apparatus for identifying scales and scale information combination features in electron microscope images. The method comprises: constructing a target detection model based on YOLOv8; constructing a first dataset based on four summarized scale-information combinations and two types of relative relationships, and training the target detection model based on the dataset; after training, first performing target detection on any electron microscope image based on the target detection model, then extracting target detection frames containing scale marks and text information from a target detection frame sequence to form first and second detection frame sequences, then identifying scale features based on the first detection frame sequence, and identifying scale information text based on the position of the information column in the current electron microscope image and the first and second detection frame sequences, and forming a combined feature from the scale features and the scale information text. The present invention can improve recognition efficiency and quality.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for identifying the scale of an electron microscope image and scale information combination features. Background Art

[0002] Electron microscopy (EM), also known as electron microscopy, is a high-resolution imaging technology widely used in materials science, biology, and medical research. The output image of an electron microscope is called an electron microscope image. While the parameters and information provided by different electron microscope manufacturers vary, all images must include a scale parameter, which consists of a pixel scale and scale information (i.e., distance unit information). Currently, most scale parameter recognition tasks for electron microscope images are still based on manual measurement. However, due to human factors and the level of manual measurement, this conventional processing method inevitably faces the problem of difficulty in improving recognition efficiency and quality. Summary of the Invention

[0003] The purpose of the present invention is to address the defects of the existing technology and provide a method, device, electronic device and computer-readable storage medium for identifying the scale and scale information combination features of electron microscope images. The present invention preliminarily constructs an electron microscope image target detection model based on the YOLOv8 model, which is recorded as a first target detection model, and summarizes the combined features of the scale-information combination in the electron microscope image to obtain four types of scale-information combinations, and summarizes the relative position relationship between the scale-information combination and the information bar in the electron microscope image to obtain two types of relative relationships, and constructs a first data set with the four types of scale-information combinations and the two types of relative relationships as training image construction conditions, and trains the first target detection model based on the first data set; then, after the model training is completed, target detection is first performed on any electron microscope image based on the first target detection model, and target detection frames of the scale mark type and the text information type in the target detection frame sequence are respectively extracted to form a first detection frame sequence and a second detection frame sequence, and then the scale feature (including the total number of scale marks and the scale pixel length) is identified based on the first detection frame sequence, and then the information bar position is identified for the current electron microscope image, and the scale information text (including numbers and distance units) is identified based on the information bar position and the first and second detection frame sequences, and the scale feature and the scale information text form a corresponding combination feature. On the one hand, the present invention can automatically identify the scale parameters of electron microscope images, thereby improving recognition efficiency and recognition quality; on the other hand, by constructing a diverse and enhanced training data set to train the model, the generalization of the model can be improved and the stability of recognition quality can be guaranteed.

[0004] To achieve the above-mentioned object, a first aspect of an embodiment of the present invention provides a method for identifying scales and scale information combination features of an electron microscope image, the method comprising:

[0005] Based on the YOLOv8 model, an electron microscope image target detection model is constructed and recorded as the corresponding first target detection model; and the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image are summarized to obtain the corresponding four types of ruler-information combinations; and the relative position relationship between the ruler-information combination and the information column in the electron microscope image is summarized to obtain the corresponding two types of relative relationships; and the corresponding model training data set is constructed using the four types of ruler-information combinations and the two types of relative relationships as training image construction conditions to obtain the corresponding first data set; and the first target detection model is trained based on the first data set;

[0006] After the model training is completed, any electron microscopy image is received as the corresponding first electron microscopy image;

[0007] performing target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence; the first target detection frame sequence includes multiple first target detection frames; the first target detection frame includes first detection frame center point coordinates, first detection frame width, first detection frame height, first detection frame type, and first detection type confidence; the first detection frame type includes a ruler mark type, a text information type, and other types;

[0008] extracting the first target detection frames whose first detection frame type is a ruler mark type from the first target detection frame sequence to form a corresponding first detection frame sequence, and extracting the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence;

[0009] Based on the first detection frame sequence, ruler feature recognition is performed to obtain a corresponding first ruler feature; and the information column position is recognized on the first electron microscope image to obtain a corresponding first information column position; and based on the first information column position, the first detection frame sequence, and the second detection frame sequence, ruler information text recognition is performed to obtain a corresponding first ruler information text; and the first ruler feature and the first ruler information text form a corresponding first combined feature.

[0010] Preferably, the scale-information combination in the electron microscope image consists of the scale and the scale information; wherein the scale contains one or more scale marks in the horizontal direction of the image, and all scale marks of the scale have the same shape, type, size and color; and the scale information is a text information containing numbers and distance units;

[0011] The four types of ruler-information combinations summarized include a first type of combination, a second type of combination, a third type of combination, and a fourth type of combination; the combination feature of the first type of combination is: the ruler includes two ruler marks, and the width of the ruler mark is less than the height, and the two ruler marks are connected by a horizontal line, and the ruler information is located at the center of the horizontal line; the combination feature of the second type of combination is: the ruler includes two ruler marks, and the width of the ruler mark is less than the height, and the two ruler marks are connected by a horizontal line, and the ruler information is above or below the horizontal line; the combination feature of the third type of combination is: the ruler includes more than two ruler marks, and the width of the ruler mark is less than the height, and there is no horizontal connecting line between every two adjacent ruler marks, and the ruler information is above or below the ruler; the combination feature of the fourth type of combination is: the ruler includes only one ruler mark, and the width of the ruler mark is greater than the height, and the ruler information is above or below the ruler;

[0012] The two types of relative relationships summarized include a first type of relative relationship and a second type of relative relationship; the first type of relative relationship means that the ruler-information combination is within the range of the information column; the second type of relative relationship means that the ruler-information combination is outside the range of the information column;

[0013] The first target detection model is used to perform corresponding ruler mark and text information target detection processing on the model input electron microscope image and output a corresponding target detection frame sequence; the target detection frame sequence includes multiple target detection frames; each target detection frame includes the coordinates of the detection frame center point, the detection frame width, the detection frame height, the detection frame type and the detection type confidence; the detection frame type includes a ruler mark type, a text information type and other types;

[0014] The first data set includes multiple first data records; the first data record includes a first training electron microscope image and a first label detection frame sequence; the first label detection frame sequence includes multiple first label detection frames; the first label detection frame includes the coordinates of the center point of the first label frame, the width of the first label frame, the height of the first label frame, the type of the first label frame, and the confidence of the first label type; the first label frame type includes a ruler mark type, a text information type, and other types; the confidence of the first label type is 1;

[0015] The first scale feature includes a total number of first scale marks and a first scale pixel length.

[0016] Preferably, the constructing of a corresponding model training data set using the four types of scale-information combinations and the two types of relative relationships as training image construction conditions to obtain a corresponding first data set specifically includes:

[0017] Step 31, pre-collecting a plurality of original electron microscope images without scales, any text information, and no information bar to form a corresponding first original electron microscope image set;

[0018] Wherein, the first original electron microscope image set includes a plurality of first original electron microscope images;

[0019] Step 32: taking the first of the first original electron microscopic images in the first original electron microscopic image set as the corresponding current bottom electron microscopic image;

[0020] Step 33, based on the random electron microscopic image generation rule, performing random electron microscopic image generation processing according to the preset first specified number N and the current bottom electron microscopic image to obtain corresponding N generated electron microscopic images;

[0021] The random electron microscope image generation rule requirements are as follows:

[0022] For each generated electron microscope image, firstly, one of the two relative relationships is randomly selected as the corresponding relative relationship type; then, the relative relationship type is identified;

[0023] If the relative relationship type is the first type of relative relationship, randomly select an information bar sub-image from the preset information bar sub-image set and add it to the current underlying electron microscope image; and the area covered by the current information bar sub-image on the current underlying electron microscope image is recorded as the corresponding current information bar area, and the area not covered by the current information bar sub-image is recorded as the corresponding current other area; and randomly add one or more random text boxes in the current other area, and randomly set the text box background color, text box transparency, text box font, text box font color, text box text and text box position in each of the random text boxes; and randomly select an integer from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, select the manufacturer's LOGO from the preset manufacturer's LOGO. Randomly select a manufacturer's LOGO image from the image set as the corresponding current manufacturer's LOGO image, and randomly select R positions on the current underlying electron microscope image to add R random image frames of random size that can adaptively adjust the image within the frame, and set the image within each of the random image frames to the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, then select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position in the current information column area; and use the electron microscope image obtained after completing all the additions this time as a corresponding generated electron microscope image;

[0024] If the relative relationship type is the second type of relative relationship, one or more random text boxes are randomly added to the current underlying electron microscope image, and the text box background color, text box transparency, text box font, text box font color, text box text and text box position in each random text box are randomly set; an integer is randomly selected from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, a manufacturer LOGO image is randomly selected from the manufacturer LOGO image set as the corresponding current manufacturer LOGO image, and a random integer is randomly selected on the current underlying electron microscope image. Add R random image frames of random size and capable of adaptively adjusting the images within the frames to R positions, and set the images within each random image frame to the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position of the current underlying electron microscope image; and use the electron microscope image obtained after completing all the additions this time as a corresponding generated electron microscope image;

[0025] The information bar sub-image set includes a plurality of the information bar sub-images; the image backgrounds of all the information bar sub-images are monochrome; the foreground image information of all the information bar sub-images includes a series of text information and the foreground text information does not contain any image information related to the ruler and ruler information; the foreground image information of some of the information bar sub-images also includes one or more horizontal and / or vertical dividing lines; the ratio of foreground pixels to background pixels of all the information bar sub-images is less than 1; when adding the information bar sub-image to the current underlying electron microscope image, the size of the information bar sub-image is scaled proportionally according to the width ratio of the information bar sub-image to the current underlying electron microscope image, and the scaled information bar sub-image is added to the bottom or top of the current underlying electron microscope image;

[0026] The manufacturer LOGO image set includes multiple manufacturer LOGO images;

[0027] The first type of ruler-information graph set includes a plurality of the first type of ruler-information graphs, and the rulers and ruler information features corresponding to the first type of ruler-information graphs satisfy the first type of combination; the second type of ruler-information graph set includes a plurality of the second type of ruler-information graphs, and the rulers and ruler information features corresponding to the second type of ruler-information graphs satisfy the second type of combination; the third type of ruler-information graph set includes a plurality of the third type of ruler-information graphs, and the rulers and ruler information features corresponding to the third type of ruler-information graphs satisfy the third type of combination; the fourth type of ruler-information graph set includes a plurality of the fourth type of ruler-information graphs, and the rulers and ruler information features corresponding to the fourth type of ruler-information graphs satisfy the fourth type of combination;

[0028] When adding the first, second, third or fourth type of ruler-information graphic to the current information bar area, appropriately resizing and moving the currently added ruler-information graphic to ensure that the currently added ruler-information graphic does not overlap with the foreground text information of the current information bar sub-image and does not exceed the current information bar area;

[0029] When adding the first, second, third or fourth type of ruler-information image to the current underlying electron microscope image, appropriately scaling and moving the currently added ruler-information image to ensure that the currently added ruler-information image does not overlap with any of the random text boxes and any of the random image boxes on the current underlying electron microscope image, and does not exceed the image range of the current underlying electron microscope image;

[0030] Step 34, perform a round of traversal on the N generated electron microscope images obtained this time; and in this round of traversal, use the generated electron microscope image currently traversed as the corresponding current electron microscope image; and mark a corresponding first label detection frame based on each of the random text boxes on the current electron microscope image, and set the first label box center point coordinates, the first label box width and the first label box height of the corresponding first label detection frame based on the text box center point coordinates, text box width and text box height of each of the random text boxes, and set the first label box type corresponding to each of the random text boxes to the text information type, and set each of the random text boxes to the first label box type. The confidence of the first label type corresponding to the text box is set to 1; and when the number of the random image frames on the current electron microscope image is not zero, a corresponding first label detection frame is marked based on each random image frame, and the first label frame center point coordinates, the first label frame width and the first label frame height of the corresponding first label detection frame are set based on the image frame center point coordinates, the image frame width and the image frame height of each random image frame, and the first label frame type corresponding to each random image frame is set to other types, and the first label type confidence corresponding to each random image frame is set to 1; and when the relative relationship type corresponding to the current electron microscope image is the first type of relative relationship, each continuous text information on the information column sub-image in the current electron microscope image is marked with a corresponding label detection frame by manual marking or other machine marking methods to obtain a plurality of corresponding first label detection frames whose first label frame type is set to the text information type and whose first label type confidence is set to 1; and the ruler information text in the unique ruler-information image on the current electron microscope image is marked as a first ruler detection frame whose first label frame type is set to the text information type and whose first label type confidence is set to 1 by manual marking or other machine marking methods. label detection frame; and mark each ruler mark sub-image area in the unique ruler-information image on the current electron microscope image into a first label detection frame in which the first label frame type is set to a ruler mark type and the first label type confidence is set to 1 by manual marking or other machine marking methods; and use the current electron microscope image as a corresponding first training electron microscope image; and form a corresponding first label detection frame sequence from all the first label detection frames corresponding to the current electron microscope image; and form a corresponding first data record from the first training electron microscope image and the first label detection frame sequence corresponding to the current electron microscope image;

[0031] Step 35: After a round of traversal of the N generated electron microscopic images obtained this time is completed, it is determined whether the current bottom-level electron microscopic image is the last first original electron microscopic image in the first original electron microscopic image set; if so, the process proceeds to step 36; if not, the next first original electron microscopic image in the first original electron microscopic image set is extracted as the new current bottom-level electron microscopic image and the process returns to step 33;

[0032] Step 36: perform data enhancement processing on each of the currently obtained first training electron micrographs based on a preset data enhancement rule to obtain a corresponding plurality of first enhanced electron micrographs; and use each of the first enhanced electron micrographs as a corresponding current enhanced electron micrograph, and use the first training electron micrograph corresponding to the current enhanced electron micrograph as a corresponding current parent image, and use the first label detection frame sequence corresponding to the current parent image as a corresponding parent image label detection frame sequence; and based on the spatial transformation relationship between the current enhanced electron micrograph and the current parent image, adaptively adjust the first label frame center point coordinates, the first label frame width, and the first label frame height of each of the first label detection frames in the parent image label detection frame sequence, and use the label detection frame sequence obtained after the adjustment as a new first label detection frame sequence, and use the current enhanced electron micrograph as a new first training electron micrograph, and form a new first data record by a pair of new first training electron micrographs and the first label detection frame sequence corresponding to the current enhanced electron micrograph;

[0033] The data enhancement rules include at least a horizontal flip enhancement rule, an HSV color transformation enhancement rule, and a low-angle rotation enhancement rule;

[0034] Step 36: All the first data records finally obtained form the corresponding first data set.

[0035] Preferably, the training of the first object detection model based on the first data set specifically includes:

[0036] Step 41: randomly split the first data set into two data subsets based on a preset first split ratio and record them as a corresponding first training set and a first evaluation set;

[0037] Wherein, both the first training set and the first evaluation set are composed of a plurality of the first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio;

[0038] Step 42: extract the first first data record from the first training set as the corresponding current training record;

[0039] Step 43: bring the first training electron microscope image of the current training record into the first target detection model to perform corresponding scale marking and text information target detection processing to obtain a corresponding first predicted target detection frame sequence;

[0040] Step 44: subjecting the first predicted object detection frame sequence and the first label detection frame sequence of the current training record to a preset first model loss function; and performing a round of optimization processing on the model parameters of the first object detection model in a direction that minimizes the first model loss function based on a preset first model optimizer;

[0041] The first model loss function is implemented based on the model loss function of the YOLOv8 model; the first model optimizer includes at least an SGD optimizer and an ADAM optimizer;

[0042] Step 45: Identify whether the current training record is the last first data record in the first training set; if so, proceed to step 46; if not, extract the next first data record in the first training set as the new current training record and return to step 43;

[0043] Step 46: perform a round of traversal on all the first data records of the first evaluation set; and during this round of traversal, use the first data record currently traversed as the corresponding current evaluation record; and bring the first training electron microscope image of the current evaluation record into the first target detection model for corresponding scale marking and text information target detection processing to obtain a corresponding second predicted target detection frame sequence; and form a corresponding first prediction-label pair from the second predicted target detection frame sequence and the first label detection frame sequence of the current training record; and at the end of this round of traversal, perform corresponding accuracy, precision, recall and F1 score calculations based on all the obtained first prediction-label pairs using the model performance evaluation method of the YOLOv8 model to obtain corresponding first accuracy, first precision, first recall and first F1 score;

[0044] Step 47: Based on the preset first accuracy range, first precision range, first recall range and first F1 score range, compare the obtained first accuracy, first precision, first recall and first F1 score; if the first accuracy does not satisfy the corresponding first accuracy range or the first precision does not satisfy the corresponding first precision range or the first recall does not satisfy the corresponding first recall range or the first F1 score does not satisfy the corresponding first F1 score range, return to step 42 to continue training; if the first accuracy satisfies the corresponding first accuracy range and the first precision satisfies the corresponding first precision range and the first recall satisfies the corresponding first recall range and the first F1 score satisfies the corresponding first F1 score range, stop training and confirm that the model training is completed.

[0045] Preferably, the performing scale feature recognition based on the first detection frame sequence to obtain the corresponding first scale feature specifically includes:

[0046] Step 51: extracting sub-images covered by each of the first target detection frames of the first detection frame sequence on the first electron microscope image as corresponding first detection frame sub-images; and performing binary image conversion on each of the first detection frame sub-images to obtain a corresponding first binary image;

[0047] The first binary image includes a plurality of first pixel points; the pixel values ​​of the first pixel points include preset foreground pixel values ​​and background pixel values; the first pixel points whose pixel values ​​are the foreground pixel values ​​are recorded as corresponding first ruler mark pixel points, and the first pixel points whose pixel values ​​are the background pixel values ​​are recorded as corresponding first background pixel points;

[0048] Step 52: copy the first electron microscope image to obtain a corresponding first copied electron microscope image; first, reset the pixel values ​​of all pixels in the first copied electron microscope image to the background pixel values; then, reset the pixel values ​​of the pixels in the first copied electron microscope image corresponding to the first scale mark pixels of each of the first binary images to the foreground pixel values; and use the first copied electron microscope image with the background and foreground pixel settings completed as the corresponding second binary image;

[0049] The second binary image includes a plurality of second pixel points; the pixel values ​​of the second pixel points include the foreground pixel value and the background pixel value; the second pixel point having the foreground pixel value is recorded as the corresponding second scale mark pixel point, and the second pixel point having the background pixel value is recorded as the corresponding second background pixel point;

[0050] Step 53: Calculate the horizontal gradient of the second binary image based on a preset gradient operator to obtain a corresponding first gradient image; calculate the sum of the pixel values ​​of each column of the first gradient image to obtain the corresponding first column pixel value sum; take the absolute value of the sum of each first column pixel value to obtain the corresponding first column absolute value; construct a two-dimensional coordinate plane with the absolute value of the pixel value sum as the vertical axis and the column index as the horizontal axis as the corresponding first coordinate plane; mark the corresponding coordinate points on the first coordinate plane based on the obtained first column absolute values ​​to obtain corresponding first marked points; connect all the obtained first marked points in sequence to obtain a corresponding first marked curve; identify the curve peak points of the first marked curve to obtain corresponding multiple first peak points; use the leftmost and rightmost first peak points as the corresponding starting peak point and ending peak point; use the integer column index closest to the starting and ending peak points on the first coordinate plane as the corresponding starting column index and ending column index; and use the difference between the ending column index and the starting column index as the corresponding first scale pixel length;

[0051] The first gradient map includes a plurality of third pixel points; the third pixel points correspond to the second pixel points one by one; the pixel value of the third pixel point corresponds to the horizontal gradient value of the corresponding second pixel point;

[0052] Step 54: Count the total number of the first peak points to obtain a corresponding first peak total number; and calculate the corresponding first scale mark total number based on the first peak total number, where the first scale mark total number = first peak total number / 2.

[0053] Step 55: Obtain the first scale feature corresponding to the total number of the first scale marks and the thickest pixel length of the first scale.

[0054] Preferably, the step of performing information column position identification on the first electron microscope image to obtain the corresponding first information column position specifically includes:

[0055] Step 61: record the width of the first electron microscope image as width W; and extract the pixel values ​​of all pixels in each row of the first electron microscope image and sort them in descending order to form a corresponding first pixel value sequence;

[0056] The first pixel value sequence consists of a plurality of first pixel values ​​p i Composition, 1≤index i≤W;

[0057] Step 62, identifying the parity of the width W, setting the corresponding 50th percentile index a=(W+1) / 2 if the width W is odd, and setting the corresponding 50th percentile index a=W / 2 if the width W is even; and setting the corresponding 5th percentile index b=floor(W×5 / 100), and the 95th percentile index c=floor(W×95 / 100), where floor() is a floor rounding function;

[0058] Step 63: In each of the first pixel value sequences, the first pixel value p whose index i matches the 50th percentile index a, the 5th percentile index b, and the 95th percentile index c is i＝a , the first pixel value p i＝b and the first pixel value p i＝c Extracted as the corresponding 50th percentile pixel value d 50 , 5th percentile pixel value d5 and 95th percentile pixel value d 95 ; and based on the 50th percentile pixel value d 50 , the 5th percentile pixel value d5 and the 95th percentile pixel value d 95 Set the corresponding first line parameter e=min(|d5-d 50 |,|d 50 -d 95 |);

[0059] Step 64: Check whether each of the first row parameters e is less than a preset first parameter threshold e. hold Identify; if the current first row parameter e is less than the first parameter threshold e hold , then the row corresponding to the first row parameter e in the first electron microscope image is recorded as the information column row;

[0060] Step 65, identifying the total number of the obtained information column rows; if the total number of the obtained information column rows is zero, setting the corresponding first information column position to be empty; if the total number of the obtained information column rows is greater than zero, identifying the maximum and minimum number of rows in all the obtained information column rows to obtain the corresponding maximum number of information column rows and minimum number of information column rows, and merging all rows from the minimum number of information column rows to the maximum number of information column rows in the first electron microscope image and using the obtained merged area as the corresponding first information column area, and extracting the four vertex coordinates of the first information column area in the first electron microscope image to form the corresponding first information column position.

[0061] Preferably, the performing ruler information text recognition based on the first information field position, the first detection frame sequence, and the second detection frame sequence to obtain the corresponding first ruler information text specifically includes:

[0062] Step 71: Identify whether the first information column position is empty. If the first information column position is not empty, record the image area covered by the first information column position in the first electron microscope image as the corresponding current information column area, and record the first target detection frame in the second detection frame sequence whose center point coordinates are within the current information column area as the corresponding candidate detection frame. If the first information column position is empty, record all the first target detection frames in the second detection frame sequence as the corresponding candidate detection frames.

[0063] Step 72: Calculate the mean of the first detection frame center coordinates of all the first target detection frames in the first detection frame sequence and use the calculated result as the corresponding first scale center coordinates; calculate the straight-line distance between the first detection frame center coordinates and the first scale center coordinates of each of the candidate detection frames to obtain a corresponding first distance; and sort the candidate detection frames in ascending order of the first distances to form a corresponding candidate detection frame sequence;

[0064] Step 73: extract the first of the candidate detection frames in the sequence of candidate detection frames as the corresponding current candidate detection frame;

[0065] Step 74: extracting the sub-image area covered by the current to-be-selected detection frame from the first electron microscope image as the corresponding current sub-image; and performing text information recognition on the current sub-image based on a preset optical character recognition tool to obtain a corresponding first recognized text;

[0066] Step 75: Identify whether the first recognition text contains numbers and distance units. If the first recognition text contains both numbers and distance units, set the corresponding first scale information text as the first recognition text, and go to step 76. If the first recognition text does not contain numbers or distance units, identify whether the current to-be-selected detection frame is the last to-be-selected detection frame in the sequence of to-be-selected detection frames. If so, set the corresponding first scale information text to be empty and go to step 76. If not, extract the next to-be-selected detection frame in the sequence of to-be-selected detection frames as the new current to-be-selected detection frame and return to step 74.

[0067] Step 76: Output the first scale information text finally obtained as the recognition result.

[0068] A second aspect of an embodiment of the present invention provides a device for implementing the method for identifying the scale and scale information combination features of an electron microscope image described in the first aspect, the device comprising: a model building and training module, an electron microscope image receiving module, a target detection module, a detection frame clustering module, and a combination feature recognition module;

[0069] The model construction and training module is used to construct an electron microscope image target detection model based on the YOLOv8 model, which is recorded as the corresponding first target detection model; and summarize the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image to obtain four corresponding scale-information combinations; and summarize the relative position relationship between the scale-information combination and the information column in the electron microscope image to obtain two corresponding relative relationships; and use the four types of scale-information combinations and the two types of relative relationships as training image construction conditions to construct a corresponding model training data set to obtain a corresponding first data set; and train the first target detection model based on the first data set;

[0070] The electron microscope image receiving module is used to receive any electron microscope image as the corresponding first electron microscope image after the model training is completed;

[0071] The target detection module is configured to perform target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence; the first target detection frame sequence includes a plurality of first target detection frames; the first target detection frame includes first detection frame center point coordinates, first detection frame width, first detection frame height, first detection frame type, and first detection type confidence; the first detection frame type includes a ruler mark type, a text information type, and other types;

[0072] The detection frame clustering module is configured to extract the first target detection frames whose first detection frame type is a ruler mark type from the first target detection frame sequence to form a corresponding first detection frame sequence, and extract the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence;

[0073] The combined feature recognition module is used to perform scale feature recognition based on the first detection frame sequence to obtain a corresponding first scale feature; and perform information column position recognition on the first electron microscope image to obtain a corresponding first information column position; and perform scale information text recognition based on the first information column position, the first detection frame sequence, and the second detection frame sequence to obtain a corresponding first scale information text; and the first scale feature and the first scale information text form a corresponding first combined feature.

[0074] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0075] The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0076] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0077] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0078] Embodiments of the present invention provide a method, device, electronic device, and computer-readable storage medium for identifying scales and scale information combination features of electron microscope images. As can be seen from the above content, the embodiment of the present invention preliminarily constructs an electron microscope image target detection model based on the YOLOv8 model, recorded as the first target detection model, and summarizes the combined features of the ruler-information combination in the electron microscope image to obtain four types of ruler-information combinations, and summarizes the relative position relationship between the ruler-information combination and the information bar in the electron microscope image to obtain two types of relative relationships, and constructs a first data set with the four types of ruler-information combinations and the two types of relative relationships as training image construction conditions, and trains the first target detection model based on the first data set; then, after the model training is completed, target detection is first performed on any electron microscope image based on the first target detection model, and then the target detection frames of the ruler mark type and the text information type in the target detection frame sequence are respectively extracted to form the first and second detection frame sequences, and then the ruler features (including the total number of ruler marks and the ruler pixel length) are identified based on the first detection frame sequence, and then the information bar position is identified for the current electron microscope image, and the ruler information text (including numbers and distance units) is identified based on the information bar position and the first and second detection frame sequences, and the ruler features and the ruler information text form a corresponding combination feature. The embodiments of the present invention can automatically identify the scale parameters of electron microscope images, thereby improving recognition efficiency and recognition quality. The embodiments of the present invention train the model by constructing a diverse and enhanced training data set, thereby improving the generalization of the model and ensuring the stability of the recognition quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram of a method for identifying scales and scale information combination features of electron microscope images provided in a first embodiment of the present invention;

[0080] Figure 2 A schematic diagram of a ruler, ruler information, and information bar provided in the first embodiment of the present invention;

[0081] Figure 3 A schematic diagram of four types of scale-information combination examples and two types of relative relationship examples provided in the first embodiment of the present invention;

[0082] Figure 4 Schematic diagrams of two examples of generating electron microscope images provided in Example 1 of the present invention;

[0083] Figure 5 A module structure diagram of a device for identifying the scale and scale information combination features of an electron microscope image provided by the second embodiment of the present invention;

[0084] Figure 6 This is a structural diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0085] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0086] The first embodiment of the present invention provides a method for identifying the scale and scale information combination features of an electron microscope image, such as Figure 1 A schematic diagram of a method for identifying scales and scale information combination features of electron microscope images provided in the first embodiment of the present invention is shown. The method mainly includes the following steps:

[0087] Step 1: Based on the YOLOv8 model, an electron microscope image target detection model is constructed, which is recorded as the corresponding first target detection model; and the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image are summarized to obtain the corresponding four types of ruler-information combinations; and the relative position relationship between the ruler-information combination and the information column in the electron microscope image is summarized to obtain the corresponding two types of relative relationships; and the four types of ruler-information combinations and the two types of relative relationships are used as training image construction conditions to construct a corresponding model training data set to obtain the corresponding first data set; and the first target detection model is trained based on the first data set;

[0088] Specifically comprising: step 11, building an electron microscope image target detection model based on the YOLOv8 model and recording it as the corresponding first target detection model;

[0089] Here, the scale-information combination in the electron microscope image of the embodiment of the present invention is composed of a scale and scale information; wherein the scale includes one or more scale marks in the horizontal direction of the image, and all scale marks of the scale have the same shape, type, size, and color; and the scale information is text information including numbers and distance units. Figure 2 This is a schematic diagram of the ruler, ruler information and information bar provided in the first embodiment of the present invention. The ruler mark, ruler, ruler information, etc. can be referred to as Figure 2 To understand;

[0090] The first target detection model of the embodiment of the present invention is used to perform corresponding ruler mark and text information target detection processing on the model input electron microscope image and output a corresponding target detection frame sequence; wherein the target detection frame sequence includes multiple target detection frames; each target detection frame includes the coordinates of the detection frame center point, the detection frame width, the detection frame height, the detection frame type, and the detection type confidence; the detection frame type includes a ruler mark type, a text information type, and other types;

[0091] Step 12, summarizing the combined features of the scale-information combination consisting of the scale and the scale information in the electron microscope image to obtain the corresponding four types of scale-information combinations;

[0092] Here, the four types of scale-information combinations summarized in the embodiment of the present invention include the first type of combination, the second type of combination, the third type of combination and the fourth type of combination; wherein,

[0093] The combination characteristics of the first type of combination are: the ruler includes two ruler marks, the width of the ruler mark is smaller than the height, the two ruler marks are connected by a horizontal line, and the ruler information is at the center of the horizontal line;

[0094] The combination characteristics of the second type of combination are: the ruler includes two ruler marks, the width of the ruler mark is smaller than the height, the two ruler marks are connected by a horizontal line, and the ruler information is above or below the horizontal line;

[0095] The combination characteristics of the third type of combination are: the ruler includes more than two ruler marks, the width of the ruler mark is smaller than the height, there is no horizontal connecting line between every two adjacent ruler marks, and the ruler information is above or below the ruler;

[0096] The fourth type of combination has the following characteristics: the ruler includes only one ruler mark, the width of the ruler mark is greater than the height, and the ruler information is above or below the ruler;

[0097] Figure 3 This is a schematic diagram of four types of ruler-information combination examples and two types of relative relationship examples provided in the first embodiment of the present invention. The above four types of ruler-information combinations can be referred to Figure 3 To understand;

[0098] Step 13, summarizing the relative positional relationship between the scale-information combination and the information column in the electron microscope image to obtain two corresponding relative relationships;

[0099] Here, the two types of relative relationships summarized in the embodiment of the present invention include a first type of relative relationship and a second type of relative relationship; the first type of relative relationship indicates that the ruler-information combination is within the range of the information column; the second type of relative relationship indicates that the ruler-information combination is outside the range of the information column; here, the information column can refer to Figure 2 To understand, the two types of relative relationships can be referred to Figure 3 To understand;

[0100] Step 14, constructing a corresponding model training data set using the four types of scale-information combinations and the two types of relative relationships as training image construction conditions to obtain a corresponding first data set;

[0101] The first data set includes multiple first data records; the first data record includes a first training electron microscope image and a first label detection frame sequence; the first label detection frame sequence includes multiple first label detection frames; the first label detection frame includes the coordinates of the center point of the first label frame, the width of the first label frame, the height of the first label frame, the type of the first label frame, and the confidence of the first label type; the first label frame type includes a ruler mark type, a text information type, and other types; the confidence of the first label type is 1;

[0102] Specifically, it includes: step 141, pre-collecting a plurality of original electron microscope images without scales, any text information, and no information bar to form a corresponding first original electron microscope image set;

[0103] The first original electron microscope image set includes a plurality of first original electron microscope images;

[0104] Step 142, taking the first first original electron microscopic image in the first original electron microscopic image set as the corresponding current bottom electron microscopic image;

[0105] Step 143 , based on the random electron microscopic image generation rule, performing random electron microscopic image generation processing according to the preset first specified number N and the current bottom electron microscopic image to obtain corresponding N generated electron microscopic images;

[0106] Here, the first specified number N is a preset positive integer;

[0107] The random electron microscope image generation rule of the embodiment of the present invention requires that when generating each generated electron microscope image:

[0108] Step A1: first randomly select one of the two relative relationships as the corresponding relative relationship type;

[0109] Step A2, identifying the relative relationship type;

[0110] Step A3, if the relative relationship type is the first type of relative relationship, randomly select an information bar sub-image from the preset information bar sub-image set and add it to the current underlying electron microscope image; and the area covered by the current information bar sub-image on the current underlying electron microscope image is recorded as the corresponding current information bar area, and the area not covered by the current information bar sub-image is recorded as the corresponding current other area; and randomly add one or more random text boxes in the current other area, and randomly set the text box background color, text box transparency, text box font, text box font color, text box text and text box position in each random text box; and randomly select an integer from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, select an integer from the preset manufacturer L Randomly select a manufacturer's LOGO image from the OGO image set as the corresponding current manufacturer's LOGO image, and randomly select R positions on the current underlying electron microscope image to add R random image frames of random size that can adaptively adjust the image within the frame, and set the image within each random image frame as the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, then select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position in the current information bar area; and use the electron microscope image obtained after completing all the additions as a corresponding generated electron microscope image;

[0111] Here, with Figure 4 Taking the two schematic diagrams of generating electron microscope images provided in the first embodiment of the present invention as an example, Figure 4 The generated electron microscope image A in the figure is a generated electron microscope image obtained when the relative relationship type is the first type of relative relationship;

[0112] Step A4: If the relative relationship type is the second type of relative relationship, one or more random text boxes are randomly added to the current underlying electron microscope image, and the text box background color, text box transparency, text box font, text box font color, text box text, and text box position in each random text box are randomly set; and an integer is randomly selected from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, a manufacturer logo image is randomly selected from the manufacturer logo image set as the corresponding current manufacturer logo image, and R is randomly selected on the current underlying electron microscope image. Add R random image frames of random size that can adaptively adjust the image inside the frame to the position, and set the image inside each random image frame to the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position of the current underlying electron microscope image; and use the electron microscope image obtained after completing all the additions this time as a corresponding generated electron microscope image;

[0113] Here, with Figure 4 For example, Figure 4 The generated electron microscope image B in the figure is a generated electron microscope image obtained when the relative relationship type is the second type of relative relationship;

[0114] The information bar sub-image set includes multiple information bar sub-images; the image background of all information bar sub-images is a monochrome background; the foreground image information of all information bar sub-images includes a series of text information, and the foreground text information does not contain any image information related to the ruler and the ruler information; the foreground image information of some information bar sub-images also includes one or more horizontal and / or vertical dividing lines; the ratio of foreground pixels to background pixels of all information bar sub-images is less than 1; when adding the information bar sub-image to the current underlying electron microscope image, the size of the information bar sub-image is scaled proportionally according to the width ratio of the information bar sub-image to the current underlying electron microscope image, and the scaled information bar sub-image is added to the bottom or top of the current underlying electron microscope image;

[0115] The manufacturer LOGO image collection includes multiple manufacturer LOGO images;

[0116] The first type of ruler-information graph set includes multiple first type ruler-information graphs, and the rulers and ruler information features corresponding to the first type ruler-information graphs meet the first type of combination; the second type of ruler-information graph set includes multiple second type ruler-information graphs, and the rulers and ruler information features corresponding to the second type ruler-information graphs meet the second type of combination; the third type of ruler-information graph set includes multiple third type ruler-information graphs, and the rulers and ruler information features corresponding to the third type ruler-information graphs meet the third type of combination; the fourth type of ruler-information graph set includes multiple fourth type ruler-information graphs, and the rulers and ruler information features corresponding to the fourth type ruler-information graphs meet the fourth type of combination; after combining the first, second, and fourth types of rulers, the rulers and ruler information features corresponding to the fourth type of ruler-information graphs meet the fourth type of combination. When adding a third or fourth type of ruler-information image to the current information bar area, the ruler-information image is appropriately scaled and moved to ensure that the ruler-information image does not overlap with the foreground text information of the current information bar sub-image and does not exceed the current information bar area; when adding a first, second, third or fourth type of ruler-information image to the current underlying electron microscope image, the ruler-information image is appropriately scaled and moved to ensure that the ruler-information image does not overlap with any random text box or image box on the current underlying electron microscope image and does not exceed the image range of the current underlying electron microscope image;

[0117] Step 144, perform a round of traversal on the N generated electron microscope images obtained this time; and in this round of traversal, use the generated electron microscope image currently traversed as the corresponding current electron microscope image; and based on each random text box on the current electron microscope image, mark a corresponding first label detection frame, and set the first label frame center point coordinates, first label frame width and first label frame height of the corresponding first label detection frame based on the text box center point coordinates, text box width and text box height of each random text box, and set the first label frame type corresponding to each random text box to the text information type, and set the first label type confidence of each random text box to 1; and when the number of random image frames on the current electron microscope image is not zero, mark a corresponding first label detection frame based on each random image frame, and set the first label frame center point coordinates, first label frame width and first label frame height of the corresponding first label detection frame based on the image frame center point coordinates, image frame width and image frame height of each random image frame, and set the first label frame type corresponding to each random image frame to other types, and set the first label type confidence of each random image frame to 1; and in the current electron microscope image, When the relative relationship type corresponding to the image is a first type of relative relationship, each continuous text information on the information column sub-image in the current electron microscope image is marked with a corresponding label detection frame by manual marking or other machine marking methods to obtain a plurality of corresponding first label detection frames whose first label frame type is set to a text information type and whose first label type confidence is set to 1; and the ruler information text in the unique ruler-information image on the current electron microscope image is marked as a first label detection frame whose first label frame type is set to a text information type and whose first label type confidence is set to 1 by manual marking or other machine marking methods; and each ruler mark sub-image area in the unique ruler-information image on the current electron microscope image is marked as a first label detection frame whose first label frame type is set to a ruler mark type and whose first label type confidence is set to 1 by manual marking or other machine marking methods; and the current electron microscope image is used as a corresponding first training electron microscope image; and all the first label detection frames corresponding to the current electron microscope image form a corresponding first label detection frame sequence; and the first training electron microscope image and the first label detection frame sequence corresponding to the current electron microscope image form a corresponding first data record;

[0118] Step 145: After a round of traversal of the N generated electron micrographs obtained this time is completed, it is determined whether the current bottom electron micrograph is the last first original electron micrograph of the first original electron micrograph set; if so, the process proceeds to step 146; if not, the next first original electron micrograph of the first original electron micrograph set is extracted as the new current bottom electron micrograph and the process returns to step 143;

[0119] Step 146: perform data enhancement processing on each of the currently obtained first training electron micrographs based on a preset data enhancement rule to obtain a corresponding plurality of first enhanced electron micrographs; and use each first enhanced electron micrograph as the corresponding current enhanced electron micrograph, and use the first training electron micrograph corresponding to the current enhanced electron micrograph as the corresponding current parent image, and use the first label detection frame sequence corresponding to the current parent image as the corresponding parent image label detection frame sequence, and adaptively adjust the first label frame center point coordinates, first label frame width, and first label frame height of each first label detection frame in the parent image label detection frame sequence based on the spatial transformation relationship between the current enhanced electron micrograph and the current parent image, and use the label detection frame sequence obtained after the adjustment as a new first label detection frame sequence, and use the current enhanced electron micrograph as a new first training electron micrograph, and form a new first data record by a pair of new first training electron micrographs and the first label detection frame sequence corresponding to the current enhanced electron micrograph;

[0120] The data enhancement rules include at least a horizontal flip enhancement rule, an HSV color transformation enhancement rule, and a low-angle rotation enhancement rule;

[0121] Step 146, forming a corresponding first data set from all the first data records finally obtained;

[0122] Step 15: training a first object detection model based on the first data set;

[0123] Specifically, it includes: step 151, randomly dividing the first data set into two data subsets based on a preset first division ratio and recording them as a corresponding first training set and a first evaluation set;

[0124] The first split ratio is a preset ratio parameter, such as 8:2; the first training set and the first evaluation set both consist of a plurality of first data records; and the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first split ratio;

[0125] Step 152: extract the first data record of the first training set as the corresponding current training record;

[0126] Step 153: The first training electron microscope image of the current training record is brought into the first target detection model to perform corresponding scale marking and text information target detection processing to obtain a corresponding first predicted target detection frame sequence;

[0127] Step 154: Substitute the first predicted object detection frame sequence and the first label detection frame sequence of the current training record into a preset first model loss function; and perform a round of optimization processing on the model parameters of the first object detection model in a direction that minimizes the first model loss function based on a preset first model optimizer;

[0128] The first model loss function is implemented based on the model loss function of the YOLOv8 model; the first model optimizer includes at least an SGD optimizer and an ADAM optimizer;

[0129] Step 155 , identifying whether the current training record is the last first data record of the first training set; if so, proceeding to step 156 ; if not, extracting the next first data record of the first training set as the new current training record and returning to step 153 ;

[0130] Step 156: Perform a round of traversal on all first data records of the first evaluation set; and during this round of traversal, use the currently traversed first data record as the corresponding current evaluation record; and bring the first training electron microscope image of the current evaluation record into the first target detection model for corresponding scale marking and text information target detection processing to obtain a corresponding second predicted target detection frame sequence; and form a corresponding first prediction-label pair from the second predicted target detection frame sequence and the first label detection frame sequence of the current training record; and at the end of this round of traversal, calculate the corresponding accuracy, precision, recall rate and F1 score based on all the obtained first prediction-label pairs using the model performance evaluation method based on the YOLOv8 model to obtain the corresponding first accuracy, first precision, first recall rate and first F1 score;

[0131] Step 157: Based on the preset first accuracy range, first precision range, first recall range, and first F1 score range, the obtained first accuracy, first precision, first recall, and first F1 score are compared; if the first accuracy does not satisfy the corresponding first accuracy range, or the first precision does not satisfy the corresponding first precision range, or the first recall does not satisfy the corresponding first recall range, or the first F1 score does not satisfy the corresponding first F1 score range, then return to step 152 to continue training; if the first accuracy satisfies the corresponding first accuracy range, the first precision satisfies the corresponding first precision range, the first recall satisfies the corresponding first recall range, and the first F1 score satisfies the corresponding first F1 score range, then stop training and confirm that the model training is complete;

[0132] Here, the first accuracy range, the first precision range, the first recall range, and the first F1 score range are four preset value ranges.

[0133] Step 2: After the model training is completed, any electron microscope image is received as the corresponding first electron microscope image.

[0134] Step 3: Perform target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence;

[0135] Among them, the first target detection frame sequence includes multiple first target detection frames; the first target detection frame includes the coordinates of the center point of the first detection frame, the width of the first detection frame, the height of the first detection frame, the first detection frame type and the confidence of the first detection type; the first detection frame type includes a ruler mark type, a text information type and other types.

[0136] Step 4: Extract the first target detection frames whose first detection frame type is a ruler mark type from the first target detection frame sequence to form a corresponding first detection frame sequence, and extract the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence.

[0137] Step 5: Performing scale feature recognition based on the first detection frame sequence to obtain a corresponding first scale feature; performing information column position recognition on the first electron microscope image to obtain a corresponding first information column position; performing scale information text recognition based on the first information column position, the first detection frame sequence, and the second detection frame sequence to obtain a corresponding first scale information text; and forming a corresponding first combined feature from the first scale feature and the first scale information text.

[0138] Specifically comprising: step 51, performing scale feature recognition based on the first detection frame sequence to obtain a corresponding first scale feature;

[0139] The first scale feature includes the total number of first scale marks and the first scale pixel length;

[0140] Specifically, the steps include: step 511, extracting sub-images covered by each first target detection frame of the first detection frame sequence on the first electron microscope image as corresponding first detection frame sub-images; and performing binary image conversion on each first detection frame sub-image to obtain a corresponding first binary image;

[0141] The first binary image includes a plurality of first pixel points; the pixel values ​​of the first pixel points include preset foreground pixel values ​​and background pixel values; the first pixel point whose pixel value is the foreground pixel value is recorded as the corresponding first scale mark pixel point, and the first pixel point whose pixel value is the background pixel value is recorded as the corresponding first background pixel point;

[0142] Step 512: copy the first electron microscope image to obtain a corresponding first copied electron microscope image; first, reset the pixel values ​​of all pixels in the first copied electron microscope image to background pixel values; then, reset the pixel values ​​of the pixels in the first copied electron microscope image corresponding to the first scale mark pixels of each first binary image to foreground pixel values; and use the first copied electron microscope image with the background and foreground pixel settings as the corresponding second binary image;

[0143] The second binary image includes a plurality of second pixel points; the pixel values ​​of the second pixel points include foreground pixel values ​​and background pixel values; the second pixel point whose pixel value is the foreground pixel value is recorded as the corresponding second scale mark pixel point, and the second pixel point whose pixel value is the background pixel value is recorded as the corresponding second background pixel point;

[0144] Step 513: Calculate the horizontal gradient of the second binary image based on a preset gradient operator to obtain a corresponding first gradient image; calculate the sum of the pixel values ​​of each column of the first gradient image to obtain the corresponding sum of the pixel values ​​of the first column; take the absolute value of the sum of the pixel values ​​of each first column to obtain the corresponding first column absolute value; construct a two-dimensional coordinate plane with the absolute value of the pixel value sum as the vertical axis and the column index as the horizontal axis as the corresponding first coordinate plane; mark the corresponding coordinate points on the first coordinate plane based on the obtained absolute values ​​of each first column to obtain the corresponding first marked points; connect all the obtained first marked points in sequence to obtain the corresponding first marked curve; identify the curve peak points of the first marked curve to obtain corresponding multiple first peak points; use the leftmost and rightmost first peak points as the corresponding starting peak point and ending peak point; use the integer column index closest to the starting and ending peak points on the first coordinate plane as the corresponding starting column index and ending column index; and use the difference between the ending column index and the starting column index as the corresponding first scale pixel length;

[0145] The gradient operator includes at least a Sobel operator and a Prewitt operator; the first gradient map includes a plurality of third pixel points; the third pixel points correspond one-to-one with the second pixel points; the pixel value of the third pixel point is the horizontal gradient value of the corresponding second pixel point;

[0146] Step 514: Count the total number of first peak points to obtain the corresponding total number of first peaks; and calculate the corresponding total number of first scale marks based on the total number of first peaks, where the total number of first scale marks = the total number of first peaks / 2.

[0147] Step 515: obtaining a first scale feature corresponding to the total number of first scale marks and the thickest first scale pixel length;

[0148] Step 52, performing information column position recognition on the first electron microscope image to obtain the corresponding first information column position;

[0149] Specifically, the process includes: step 521, recording the width of the first electron microscope image as width W; extracting the pixel values ​​of all pixels in each row of the first electron microscope image and sorting them in descending order to form a corresponding first pixel value sequence;

[0150] The first pixel value sequence consists of a plurality of first pixel values ​​p i Composition, 1≤index i≤W;

[0151] Step 522: Identify the parity of the width W. If the width W is odd, set the corresponding 50th percentile index a to (W+1) / 2; if the width W is even, set the corresponding 50th percentile index a to W / 2; and set the corresponding 5th percentile index b to floor(W×5 / 100) and 95th percentile index c to floor(W×95 / 100). Floor() is a floor rounding function.

[0152] Step 523: In each first pixel value sequence, the first pixel value p whose index i matches the 50th percentile index a, the 5th percentile index b, and the 95th percentile index c is i＝a , the first pixel value p i＝b and the first pixel value p i＝c Extracted as the corresponding 50th percentile pixel value d 50 , 5th percentile pixel value d5 and 95th percentile pixel value d 95 ; and based on the 50th percentile pixel value d 50 , 5th percentile pixel value d5 and 95th percentile pixel value d 95 Set the corresponding first line parameter e=min(|d5-d 50 |,|d 50 -d 95 |);

[0153] Step 524: Check whether each first row parameter e is less than a preset first parameter threshold e. hold Identify; if the current first row parameter e is less than the first parameter threshold e hold , then the row corresponding to the first row of parameter e in the first electron microscope image is recorded as the information column row;

[0154] Here, the first parameter threshold e hold is a preset threshold parameter;

[0155] Step 525: Identify the total number of information column rows obtained; if the total number of information column rows obtained is zero, set the corresponding first information column position to be empty; if the total number of information column rows obtained is greater than zero, identify the maximum and minimum number of rows among all the obtained information column rows to obtain the corresponding maximum and minimum number of information column rows, merge all rows between the minimum and maximum number of information column rows in the first electron microscope image, and use the resulting merged area as the corresponding first information column area. Extract the coordinates of the four vertices of the first information column area in the first electron microscope image to form the corresponding first information column position.

[0156] Step 53: performing ruler information text recognition based on the first information field position, the first detection frame sequence, and the second detection frame sequence to obtain corresponding first ruler information text;

[0157] Specifically, the method includes: step 531, identifying whether the first information column position is empty; if the first information column position is not empty, recording the image area covered by the first information column position in the first electron microscope image as the corresponding current information column area, and recording the first target detection frame in the second detection frame sequence whose center point coordinates are within the current information column area as the corresponding candidate detection frame; if the first information column position is empty, recording all first target detection frames in the second detection frame sequence as corresponding candidate detection frames;

[0158] Step 532: Calculate the mean of the first detection frame center coordinates of all first target detection frames in the first detection frame sequence and use the calculated result as the corresponding first scale center coordinates; calculate the straight-line distance between the first detection frame center coordinates of each candidate detection frame and the first scale center coordinates to obtain a corresponding first distance; and sort the candidate detection frames in ascending order of the first distances to form a corresponding candidate detection frame sequence;

[0159] Step 533: extract the first detection frame to be selected from the sequence of detection frames to be selected as the corresponding current detection frame to be selected;

[0160] Step 534: extract the sub-image area covered by the current to-be-selected detection frame from the first electron microscope image as the corresponding current sub-image; and perform text information recognition on the current sub-image using a preset optical character recognition tool to obtain a corresponding first recognized text;

[0161] Here, the optical character recognition (OCR) tool is a pre-set OCR tool, service / interface, including at least PaddleOCR, Tianruo OCR, iFlytek OCR, etc.

[0162] Step 535: Determine whether the first recognized text contains numbers and distance units. If the first recognized text contains both numbers and distance units, set the corresponding first scale information text as the first recognized text, and proceed to step 536. If the first recognized text does not contain numbers or distance units, determine whether the current candidate detection frame is the last candidate detection frame in the sequence of candidate detection frames. If so, set the corresponding first scale information text to empty and proceed to step 536. If not, extract the next candidate detection frame in the sequence of candidate detection frames as the new current candidate detection frame and return to step 534.

[0163] Step 536: Output the first scale information text finally obtained as the recognition result;

[0164] In step 54, the first scale feature and the first scale information text are combined to form a corresponding first combined feature.

[0165] Here, the first combined feature obtained by the present invention is composed of the corresponding total number of first scale marks (actually the total number of scales of the pixel scale), the first scale pixel length (actually the total length of the pixel scale) and the first scale information text. The pixel scale can be obtained from the total number of first scale marks and the first scale pixel length, and the conversion relationship between the corresponding pixel scale and the actual physical distance can be obtained from the data and units in the first scale information text.

[0166] Figure 5 This is a module structure diagram of a device for identifying the scale and scale information combination features of an electron microscope image provided in the second embodiment of the present invention. The device is a terminal device or server that implements the aforementioned method embodiment, and can also be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 5 As shown, the device includes: a model building and training module 201, an electron microscope image receiving module 202, a target detection module 203, a detection frame clustering module 204 and a combined feature recognition module 205.

[0167] The model construction and training module 201 is used to construct an electron microscope image target detection model based on the YOLOv8 model, which is recorded as the corresponding first target detection model; and summarize the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image to obtain the corresponding four types of ruler-information combinations; and summarize the relative position relationship between the ruler-information combination and the information column in the electron microscope image to obtain the corresponding two types of relative relationships; and use the four types of ruler-information combinations and the two types of relative relationships as training image construction conditions to construct a corresponding model training data set to obtain the corresponding first data set; and train the first target detection model based on the first data set.

[0168] The electron microscope image receiving module 202 is used to receive any electron microscope image as the corresponding first electron microscope image after the model training is completed.

[0169] The target detection module 203 is used to perform target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence; the first target detection frame sequence includes multiple first target detection frames; the first target detection frame includes the coordinates of the center point of the first detection frame, the width of the first detection frame, the height of the first detection frame, the first detection frame type and the confidence of the first detection type; the first detection frame type includes a ruler mark type, a text information type and other types.

[0170] The detection frame clustering module 204 is used to extract the first target detection frames whose first detection frame type is a ruler mark type in the first target detection frame sequence to form a corresponding first detection frame sequence, and extract the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence.

[0171] The combined feature recognition module 205 is used to perform scale feature recognition based on the first detection frame sequence to obtain a corresponding first scale feature; and perform information column position recognition on the first electron microscope image to obtain a corresponding first information column position; and perform scale information text recognition based on the first information column position, the first detection frame sequence, and the second detection frame sequence to obtain a corresponding first scale information text; and the first scale feature and the first scale information text form a corresponding first combined feature.

[0172] An embodiment of the present invention provides a device for identifying the scale and scale information combination features of an electron microscope image, which can execute the method steps in the above method embodiment. Its implementation principles and technical effects are similar and will not be repeated here.

[0173] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the model building and training module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above-mentioned determined modules. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0174] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0175] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the above method embodiments are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) means. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, hard disk, tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0176] Figure 6 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. The electronic device can be a terminal device or server that implements the method of the aforementioned embodiment, or it can be a terminal device or server that implements the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 6As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303's transceiver actions. Various instructions may be stored in the memory 302 for completing various processing functions and implementing the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0177] exist Figure 6 The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0178] The above-mentioned processors can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0179] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0180] Embodiments of the present invention provide a method, device, electronic device, and computer-readable storage medium for identifying scales and scale information combination features of electron microscope images. As can be seen from the above content, the embodiment of the present invention preliminarily constructs an electron microscope image target detection model based on the YOLOv8 model, recorded as the first target detection model, and summarizes the combined features of the ruler-information combination in the electron microscope image to obtain four types of ruler-information combinations, and summarizes the relative position relationship between the ruler-information combination and the information bar in the electron microscope image to obtain two types of relative relationships, and constructs a first data set with the four types of ruler-information combinations and the two types of relative relationships as training image construction conditions, and trains the first target detection model based on the first data set; then, after the model training is completed, target detection is first performed on any electron microscope image based on the first target detection model, and then the target detection frames of the ruler mark type and the text information type in the target detection frame sequence are respectively extracted to form the first and second detection frame sequences, and then the ruler features (including the total number of ruler marks and the ruler pixel length) are identified based on the first detection frame sequence, and then the information bar position is identified for the current electron microscope image, and the ruler information text (including numbers and distance units) is identified based on the information bar position and the first and second detection frame sequences, and the ruler features and the ruler information text form a corresponding combination feature. The embodiments of the present invention can automatically identify the scale parameters of electron microscope images, thereby improving recognition efficiency and recognition quality. The embodiments of the present invention train the model by constructing a diverse and enhanced training data set, thereby improving the generalization of the model and ensuring the stability of the recognition quality.

[0181] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0182] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying scale and scale information combination features of electron microscope images, characterized in that: The method comprises: Based on the YOLOv8 model, an electron microscope image target detection model is constructed and recorded as the corresponding first target detection model; and the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image are summarized to obtain the corresponding four types of ruler-information combinations; and the relative position relationship between the ruler-information combination and the information column in the electron microscope image is summarized to obtain the corresponding two types of relative relationships; and the corresponding model training data set is constructed using the four types of ruler-information combinations and the two types of relative relationships as training image construction conditions to obtain the corresponding first data set; and the first target detection model is trained based on the first data set; After the model training is completed, any electron microscopy image is received as the corresponding first electron microscopy image; performing target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence; the first target detection frame sequence includes multiple first target detection frames; the first target detection frame includes first detection frame center point coordinates, first detection frame width, first detection frame height, first detection frame type, and first detection type confidence; the first detection frame type includes a ruler mark type, a text information type, and other types; extracting the first target detection frames whose first detection frame type is a ruler mark type from the first target detection frame sequence to form a corresponding first detection frame sequence, and extracting the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence; Based on the first detection frame sequence, ruler feature recognition is performed to obtain a corresponding first ruler feature; and the information column position is recognized on the first electron microscope image to obtain a corresponding first information column position; and based on the first information column position, the first detection frame sequence, and the second detection frame sequence, ruler information text recognition is performed to obtain a corresponding first ruler information text; and the first ruler feature and the first ruler information text form a corresponding first combined feature.

2. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 1, characterized in that: The scale-information combination in the electron microscope image consists of the scale and the scale information; wherein the scale contains one or more scale marks in the horizontal direction of the image, and all scale marks of the scale have the same shape, type, size, and color; and the scale information is text information containing numbers and distance units; The four types of ruler-information combinations summarized include a first type of combination, a second type of combination, a third type of combination, and a fourth type of combination; the combination feature of the first type of combination is: the ruler includes two ruler marks, and the width of the ruler mark is less than the height, and the two ruler marks are connected by a horizontal line, and the ruler information is located at the center of the horizontal line; the combination feature of the second type of combination is: the ruler includes two ruler marks, and the width of the ruler mark is less than the height, and the two ruler marks are connected by a horizontal line, and the ruler information is above or below the horizontal line; the combination feature of the third type of combination is: the ruler includes more than two ruler marks, and the width of the ruler mark is less than the height, and there is no horizontal connecting line between every two adjacent ruler marks, and the ruler information is above or below the ruler; the combination feature of the fourth type of combination is: the ruler includes only one ruler mark, and the width of the ruler mark is greater than the height, and the ruler information is above or below the ruler; The two types of relative relationships summarized include a first type of relative relationship and a second type of relative relationship; the first type of relative relationship means that the ruler-information combination is within the range of the information column; the second type of relative relationship means that the ruler-information combination is outside the range of the information column; The first target detection model is used to perform corresponding ruler mark and text information target detection processing on the model input electron microscope image and output a corresponding target detection frame sequence; the target detection frame sequence includes multiple target detection frames; each target detection frame includes the coordinates of the detection frame center point, the detection frame width, the detection frame height, the detection frame type and the detection type confidence; the detection frame type includes a ruler mark type, a text information type and other types; The first data set includes multiple first data records; the first data record includes a first training electron microscope image and a first label detection frame sequence; the first label detection frame sequence includes multiple first label detection frames; the first label detection frame includes the coordinates of the center point of the first label frame, the width of the first label frame, the height of the first label frame, the type of the first label frame, and the confidence of the first label type; the first label frame type includes a ruler mark type, a text information type, and other types; the confidence of the first label type is 1; The first scale feature includes a total number of first scale marks and a first scale pixel length.

3. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 2, characterized in that: The constructing of a corresponding model training data set using the four types of scale-information combinations and the two types of relative relationships as training image construction conditions to obtain a corresponding first data set specifically includes: Step 31, pre-collecting a plurality of original electron microscope images without scales, any text information, and no information bar to form a corresponding first original electron microscope image set; Wherein, the first original electron microscope image set includes a plurality of first original electron microscope images; Step 32: taking the first of the first original electron microscopic images in the first original electron microscopic image set as the corresponding current bottom electron microscopic image; Step 33, based on the random electron microscopic image generation rule, performing random electron microscopic image generation processing according to the preset first specified number N and the current bottom electron microscopic image to obtain corresponding N generated electron microscopic images; The random electron microscope image generation rule requirements are as follows: For each generated electron microscope image, firstly, one of the two relative relationships is randomly selected as the corresponding relative relationship type; then, the relative relationship type is identified; If the relative relationship type is the first type of relative relationship, randomly select an information bar sub-image from the preset information bar sub-image set and add it to the current underlying electron microscope image; and the area covered by the current information bar sub-image on the current underlying electron microscope image is recorded as the corresponding current information bar area, and the area not covered by the current information bar sub-image is recorded as the corresponding current other area; and randomly add one or more random text boxes in the current other area, and randomly set the text box background color, text box transparency, text box font, text box font color, text box text and text box position in each of the random text boxes; and randomly select an integer from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, select the manufacturer's LOGO from the preset manufacturer's LOGO. Randomly select a manufacturer's LOGO image from the image set as the corresponding current manufacturer's LOGO image, and randomly select R positions on the current underlying electron microscope image to add R random image frames of random size that can adaptively adjust the image within the frame, and set the image within each of the random image frames to the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, then select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position in the current information column area; and use the electron microscope image obtained after completing all the additions this time as a corresponding generated electron microscope image; If the relative relationship type is the second type of relative relationship, one or more random text boxes are randomly added to the current underlying electron microscope image, and the text box background color, text box transparency, text box font, text box font color, text box text and text box position in each random text box are randomly set; an integer is randomly selected from the value range of 0 to the second specified number M as the corresponding current random number R, and when the current random number R is not 0, a manufacturer LOGO image is randomly selected from the manufacturer LOGO image set as the corresponding current manufacturer LOGO image, and a random integer is randomly selected on the current underlying electron microscope image. Add R random image frames of random size and capable of adaptively adjusting the images within the frames to R positions, and set the images within each random image frame to the corresponding current manufacturer's LOGO image; and select one of the four types of ruler-information combinations as the corresponding current combination; if the current combination is the first, second, third or fourth type combination, select a corresponding first, second, third or fourth type ruler-information image from the preset first, second, third or fourth type ruler-information image set and add it to an optional position of the current underlying electron microscope image; and use the electron microscope image obtained after completing all the additions this time as a corresponding generated electron microscope image; The information bar sub-image set includes a plurality of the information bar sub-images; the image backgrounds of all the information bar sub-images are monochrome; the foreground image information of all the information bar sub-images includes a series of text information and the foreground text information does not contain any image information related to the ruler and ruler information; the foreground image information of some of the information bar sub-images also includes one or more horizontal and / or vertical dividing lines; the ratio of foreground pixels to background pixels of all the information bar sub-images is less than 1; when adding the information bar sub-image to the current underlying electron microscope image, the size of the information bar sub-image is scaled proportionally according to the width ratio of the information bar sub-image to the current underlying electron microscope image, and the scaled information bar sub-image is added to the bottom or top of the current underlying electron microscope image; The manufacturer LOGO image set includes multiple manufacturer LOGO images; The first type of ruler-information graph set includes a plurality of the first type of ruler-information graphs, and the rulers and ruler information features corresponding to the first type of ruler-information graphs satisfy the first type of combination; the second type of ruler-information graph set includes a plurality of the second type of ruler-information graphs, and the rulers and ruler information features corresponding to the second type of ruler-information graphs satisfy the second type of combination; the third type of ruler-information graph set includes a plurality of the third type of ruler-information graphs, and the rulers and ruler information features corresponding to the third type of ruler-information graphs satisfy the third type of combination; the fourth type of ruler-information graph set includes a plurality of the fourth type of ruler-information graphs, and the rulers and ruler information features corresponding to the fourth type of ruler-information graphs satisfy the fourth type of combination; When adding the first, second, third or fourth type of ruler-information graphic to the current information bar area, appropriately resizing and moving the currently added ruler-information graphic to ensure that the currently added ruler-information graphic does not overlap with the foreground text information of the current information bar sub-image and does not exceed the current information bar area; When adding the first, second, third or fourth type of ruler-information image to the current underlying electron microscope image, appropriately scaling and moving the currently added ruler-information image to ensure that the currently added ruler-information image does not overlap with any of the random text boxes and any of the random image boxes on the current underlying electron microscope image, and does not exceed the image range of the current underlying electron microscope image; Step 34, perform a round of traversal on the N generated electron microscope images obtained this time; and in this round of traversal, use the generated electron microscope image currently traversed as the corresponding current electron microscope image; and mark a corresponding first label detection frame based on each of the random text boxes on the current electron microscope image, and set the first label box center point coordinates, the first label box width and the first label box height of the corresponding first label detection frame based on the text box center point coordinates, text box width and text box height of each of the random text boxes, and set the first label box type corresponding to each of the random text boxes to the text information type, and set each of the random text boxes to the first label box type. The confidence of the first label type corresponding to the text box is set to 1; and when the number of the random image frames on the current electron microscope image is not zero, a corresponding first label detection frame is marked based on each random image frame, and the first label frame center point coordinates, the first label frame width and the first label frame height of the corresponding first label detection frame are set based on the image frame center point coordinates, the image frame width and the image frame height of each random image frame, and the first label frame type corresponding to each random image frame is set to other types, and the first label type confidence corresponding to each random image frame is set to 1; and when the relative relationship type corresponding to the current electron microscope image is the first type of relative relationship, each continuous text information on the information column sub-image in the current electron microscope image is marked with a corresponding label detection frame by manual marking or other machine marking methods to obtain a plurality of corresponding first label detection frames whose first label frame type is set to the text information type and whose first label type confidence is set to 1; and the ruler information text in the unique ruler-information image on the current electron microscope image is marked as a first ruler detection frame whose first label frame type is set to the text information type and whose first label type confidence is set to 1 by manual marking or other machine marking methods. label detection frame; and mark each ruler mark sub-image area in the unique ruler-information image on the current electron microscope image into a first label detection frame in which the first label frame type is set to a ruler mark type and the first label type confidence is set to 1 by manual marking or other machine marking methods; and use the current electron microscope image as a corresponding first training electron microscope image; and form a corresponding first label detection frame sequence from all the first label detection frames corresponding to the current electron microscope image; and form a corresponding first data record from the first training electron microscope image and the first label detection frame sequence corresponding to the current electron microscope image; Step 35: After a round of traversal of the N generated electron microscopic images obtained this time is completed, it is determined whether the current bottom-level electron microscopic image is the last first original electron microscopic image in the first original electron microscopic image set; if so, the process proceeds to step 36; if not, the next first original electron microscopic image in the first original electron microscopic image set is extracted as the new current bottom-level electron microscopic image and the process returns to step 33; Step 36: perform data enhancement processing on each of the currently obtained first training electron micrographs based on a preset data enhancement rule to obtain a corresponding plurality of first enhanced electron micrographs; and use each of the first enhanced electron micrographs as a corresponding current enhanced electron micrograph, and use the first training electron micrograph corresponding to the current enhanced electron micrograph as a corresponding current parent image, and use the first label detection frame sequence corresponding to the current parent image as a corresponding parent image label detection frame sequence; and based on the spatial transformation relationship between the current enhanced electron micrograph and the current parent image, adaptively adjust the first label frame center point coordinates, the first label frame width, and the first label frame height of each of the first label detection frames in the parent image label detection frame sequence, and use the label detection frame sequence obtained after the adjustment as a new first label detection frame sequence, and use the current enhanced electron micrograph as a new first training electron micrograph, and form a new first data record by a pair of new first training electron micrographs and the first label detection frame sequence corresponding to the current enhanced electron micrograph; The data enhancement rules include at least a horizontal flip enhancement rule, an HSV color transformation enhancement rule, and a low-angle rotation enhancement rule; Step 36: All the first data records finally obtained form the corresponding first data set.

4. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 2, characterized in that: The training of the first target detection model based on the first data set specifically includes: Step 41: randomly split the first data set into two data subsets based on a preset first split ratio and record them as a corresponding first training set and a first evaluation set; Wherein, both the first training set and the first evaluation set are composed of a plurality of the first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio; Step 42: extract the first first data record from the first training set as the corresponding current training record; Step 43: bring the first training electron microscope image of the current training record into the first target detection model to perform corresponding scale marking and text information target detection processing to obtain a corresponding first predicted target detection frame sequence; Step 44: subjecting the first predicted object detection frame sequence and the first label detection frame sequence of the current training record to a preset first model loss function; and performing a round of optimization processing on the model parameters of the first object detection model in a direction that minimizes the first model loss function based on a preset first model optimizer; The first model loss function is implemented based on the model loss function of the YOLOv8 model; the first model optimizer includes at least an SGD optimizer and an ADAM optimizer; Step 45: Identify whether the current training record is the last first data record in the first training set; if so, proceed to step 46; if not, extract the next first data record in the first training set as the new current training record and return to step 43; Step 46: perform a round of traversal on all the first data records of the first evaluation set; and during this round of traversal, use the first data record currently traversed as the corresponding current evaluation record; and bring the first training electron microscope image of the current evaluation record into the first target detection model for corresponding scale marking and text information target detection processing to obtain a corresponding second predicted target detection frame sequence; and form a corresponding first prediction-label pair from the second predicted target detection frame sequence and the first label detection frame sequence of the current training record; and at the end of this round of traversal, perform corresponding accuracy, precision, recall and F1 score calculations based on all the obtained first prediction-label pairs using the model performance evaluation method of the YOLOv8 model to obtain corresponding first accuracy, first precision, first recall and first F1 score; Step 47: Based on the preset first accuracy range, first precision range, first recall range and first F1 score range, compare the obtained first accuracy, first precision, first recall and first F1 score; if the first accuracy does not satisfy the corresponding first accuracy range or the first precision does not satisfy the corresponding first precision range or the first recall does not satisfy the corresponding first recall range or the first F1 score does not satisfy the corresponding first F1 score range, return to step 42 to continue training; if the first accuracy satisfies the corresponding first accuracy range and the first precision satisfies the corresponding first precision range and the first recall satisfies the corresponding first recall range and the first F1 score satisfies the corresponding first F1 score range, stop training and confirm that the model training is completed.

5. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 2, characterized in that: The performing scale feature recognition based on the first detection frame sequence to obtain the corresponding first scale feature specifically includes: Step 51: extracting sub-images covered by each of the first target detection frames of the first detection frame sequence on the first electron microscope image as corresponding first detection frame sub-images; and performing binary image conversion on each of the first detection frame sub-images to obtain a corresponding first binary image; The first binary image includes a plurality of first pixel points; the pixel values ​​of the first pixel points include preset foreground pixel values ​​and background pixel values; the first pixel points whose pixel values ​​are the foreground pixel values ​​are recorded as corresponding first ruler mark pixel points, and the first pixel points whose pixel values ​​are the background pixel values ​​are recorded as corresponding first background pixel points; Step 52: copy the first electron microscope image to obtain a corresponding first copied electron microscope image; first, reset the pixel values ​​of all pixels in the first copied electron microscope image to the background pixel values; then, reset the pixel values ​​of the pixels in the first copied electron microscope image corresponding to the first scale mark pixels of each of the first binary images to the foreground pixel values; and use the first copied electron microscope image with the background and foreground pixel settings completed as the corresponding second binary image; The second binary image includes a plurality of second pixel points; the pixel values ​​of the second pixel points include the foreground pixel value and the background pixel value; the second pixel point having the foreground pixel value is recorded as the corresponding second scale mark pixel point, and the second pixel point having the background pixel value is recorded as the corresponding second background pixel point; Step 53: Calculate the horizontal gradient of the second binary image based on a preset gradient operator to obtain a corresponding first gradient image; calculate the sum of the pixel values ​​of each column of the first gradient image to obtain the corresponding first column pixel value sum; take the absolute value of the sum of each first column pixel value to obtain the corresponding first column absolute value; construct a two-dimensional coordinate plane with the absolute value of the pixel value sum as the vertical axis and the column index as the horizontal axis as the corresponding first coordinate plane; mark the corresponding coordinate points on the first coordinate plane based on the obtained first column absolute values ​​to obtain corresponding first marked points; connect all the obtained first marked points in sequence to obtain a corresponding first marked curve; identify the curve peak points of the first marked curve to obtain corresponding multiple first peak points; use the leftmost and rightmost first peak points as the corresponding starting peak point and ending peak point; use the integer column index closest to the starting and ending peak points on the first coordinate plane as the corresponding starting column index and ending column index; and use the difference between the ending column index and the starting column index as the corresponding first scale pixel length; The first gradient map includes a plurality of third pixel points; the third pixel points correspond to the second pixel points one by one; the pixel value of the third pixel point corresponds to the horizontal gradient value of the corresponding second pixel point; Step 54: Count the total number of the first peak points to obtain a corresponding first peak total number; and calculate the corresponding first scale mark total number based on the first peak total number, where the first scale mark total number = first peak total number / 2. Step 55: Obtain the first scale feature corresponding to the total number of the first scale marks and the thickest pixel length of the first scale.

6. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 2, characterized in that: The step of performing information column position identification on the first electron microscope image to obtain the corresponding first information column position specifically includes: Step 61: record the width of the first electron microscope image as width W; and extract the pixel values ​​of all pixels in each row of the first electron microscope image and sort them in descending order to form a corresponding first pixel value sequence; The first pixel value sequence consists of a plurality of first pixel values ​​p i Composition, 1≤index i≤W; Step 62, identifying the parity of the width W, setting the corresponding 50th percentile index a=(W+1) / 2 if the width W is odd, and setting the corresponding 50th percentile index a=W / 2 if the width W is even; and setting the corresponding 5th percentile index b=floor(W×5 / 100), and the 95th percentile index c=floor(W×95 / 100), where floor() is a floor rounding function; Step 63: In each of the first pixel value sequences, the first pixel value p whose index i matches the 50th percentile index a, the 5th percentile index b, and the 95th percentile index c is i＝a , the first pixel value p i＝b and the first pixel value p i＝c Extracted as the corresponding 50th percentile pixel value d 50 , 5th percentile pixel value d5 and 95th percentile pixel value d 95 ; and based on the 50th percentile pixel value d 50 , the 5th percentile pixel value d5 and the 95th percentile pixel value d 95 Set the corresponding first line parameter e=min(|d5-d 50 |,|d 50 -d 95 |); Step 64: Check whether each of the first row parameters e is less than a preset first parameter threshold e. hold Identify; if the current first row parameter e is less than the first parameter threshold e hold , then the row corresponding to the first row parameter e in the first electron microscope image is recorded as the information column row; Step 65, identifying the total number of the obtained information column rows; if the total number of the obtained information column rows is zero, setting the corresponding first information column position to be empty; if the total number of the obtained information column rows is greater than zero, identifying the maximum and minimum number of rows in all the obtained information column rows to obtain the corresponding maximum number of information column rows and minimum number of information column rows, and merging all rows from the minimum number of information column rows to the maximum number of information column rows in the first electron microscope image and using the obtained merged area as the corresponding first information column area, and extracting the four vertex coordinates of the first information column area in the first electron microscope image to form the corresponding first information column position.

7. The method for identifying the scale and scale information combination features of an electron microscope image according to claim 2, characterized in that: The performing ruler information text recognition based on the first information field position, the first detection frame sequence, and the second detection frame sequence to obtain the corresponding first ruler information text specifically includes: Step 71: Identify whether the first information column position is empty. If the first information column position is not empty, record the image area covered by the first information column position in the first electron microscope image as the corresponding current information column area, and record the first target detection frame in the second detection frame sequence whose center point coordinates are within the current information column area as the corresponding candidate detection frame. If the first information column position is empty, record all the first target detection frames in the second detection frame sequence as the corresponding candidate detection frames. Step 72: Calculate the mean of the first detection frame center coordinates of all the first target detection frames in the first detection frame sequence and use the calculated result as the corresponding first scale center coordinates; calculate the straight-line distance between the first detection frame center coordinates and the first scale center coordinates of each of the candidate detection frames to obtain a corresponding first distance; and sort the candidate detection frames in ascending order of the first distances to form a corresponding candidate detection frame sequence; Step 73: extract the first of the candidate detection frames in the sequence of candidate detection frames as the corresponding current candidate detection frame; Step 74: extracting the sub-image area covered by the current to-be-selected detection frame from the first electron microscope image as the corresponding current sub-image; and performing text information recognition on the current sub-image based on a preset optical character recognition tool to obtain a corresponding first recognized text; Step 75: Identify whether the first recognition text contains numbers and distance units. If the first recognition text contains both numbers and distance units, set the corresponding first scale information text as the first recognition text, and go to step 76. If the first recognition text does not contain numbers or distance units, identify whether the current to-be-selected detection frame is the last to-be-selected detection frame in the sequence of to-be-selected detection frames. If so, set the corresponding first scale information text to be empty and go to step 76. If not, extract the next to-be-selected detection frame in the sequence of to-be-selected detection frames as the new current to-be-selected detection frame and return to step 74. Step 76: Output the first scale information text finally obtained as the recognition result.

8. A device for executing the method for identifying the scale and scale information combination features of an electron microscope image according to any one of claims 1 to 7, characterized in that: The device comprises: a model building and training module, an electron microscope image receiving module, a target detection module, a detection frame clustering module and a combined feature recognition module; The model construction and training module is used to construct an electron microscope image target detection model based on the YOLOv8 model, which is recorded as the corresponding first target detection model; and summarize the combined features of the ruler-information combination composed of the ruler and the ruler information in the electron microscope image to obtain four corresponding scale-information combinations; and summarize the relative position relationship between the scale-information combination and the information column in the electron microscope image to obtain two corresponding relative relationships; and use the four types of scale-information combinations and the two types of relative relationships as training image construction conditions to construct a corresponding model training data set to obtain a corresponding first data set; and train the first target detection model based on the first data set; The electron microscope image receiving module is used to receive any electron microscope image as the corresponding first electron microscope image after the model training is completed; The target detection module is configured to perform target detection processing on the first electron microscope image based on the first target detection model to obtain a corresponding first target detection frame sequence; the first target detection frame sequence includes a plurality of first target detection frames; the first target detection frame includes first detection frame center point coordinates, first detection frame width, first detection frame height, first detection frame type, and first detection type confidence; the first detection frame type includes a ruler mark type, a text information type, and other types; The detection frame clustering module is configured to extract the first target detection frames whose first detection frame type is a ruler mark type from the first target detection frame sequence to form a corresponding first detection frame sequence, and extract the first target detection frames whose first detection frame type is a text information type to form a corresponding second detection frame sequence; The combined feature recognition module is used to perform scale feature recognition based on the first detection frame sequence to obtain a corresponding first scale feature; and perform information column position recognition on the first electron microscope image to obtain a corresponding first information column position; and perform scale information text recognition based on the first information column position, the first detection frame sequence, and the second detection frame sequence to obtain a corresponding first scale information text; and the first scale feature and the first scale information text form a corresponding first combined feature.

9. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 7; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images