Chip identifier identification method, device, computer equipment, medium and product

Abstract

The present application relates to a chip identifier recognition method, device, computer equipment, medium and product. The method comprises: obtaining a chip image; the chip image comprises at least one chip image, each chip image is an image of the chip surface where the chip identifier is located; based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence; multi-level fusion is performed according to the features extracted by at least some of the feature extraction units in the plurality of feature extraction units; based on the target fusion features obtained by the multi-level fusion, the target text area in each chip image is detected from the chip image; text recognition is performed on the target text area of ​​each chip image to obtain the identifier of the chip corresponding to each chip image. The use of this method can improve the recognition efficiency of chip identifiers.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a chip identifier recognition method, device, computer equipment, medium and product. Background Art

[0002] A chip is a microelectronic circuit that integrates a large number of tiny electronic components on a single semiconductor substrate. With the continuous improvement of chip technology, chips are widely used in various electronic devices. The chip package surface may include a chip identifier, which is important information for identifying the chip. During the production stage, the chip identifier facilitates the tracking and management of the chip. The chip manufacturer can use the chip identifier to track and manage the chip's production process, test results, quality status, etc., so as to promptly discover and resolve possible quality problems. Therefore, it is of great significance to identify the chip identifier. At present, chip identifier identification usually uses manual visual inspection, and the staff on the production line check and transcribe the chip identifier.

[0003] However, chip identifier recognition through manual visual inspection has the problem of low recognition efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a chip identifier recognition method, device, computer equipment, computer-readable storage medium and computer program product that can improve recognition efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a chip identifier recognition method, comprising:

[0006] Acquire a chip image; the chip image includes at least one chip image, each of which is an image of the chip surface where the chip identifier is located;

[0007] Based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence;

[0008] Performing multi-level fusion according to the features extracted by at least some of the feature extraction units among the multiple feature extraction units;

[0009] Based on the target fusion features obtained by the multi-level fusion, detecting the target text area in each chip image from the chip image;

[0010] Text recognition is performed on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0011] In a second aspect, the present application further provides a chip identifier recognition device, comprising:

[0012] An acquisition module, used for acquiring a chip image; the chip image includes at least one chip image, each of which is an image of a chip surface where a chip identifier is located;

[0013] A region detection module is used to extract features in sequence through a plurality of feature extraction units connected in sequence based on the chip image; perform multi-level fusion according to the features extracted by at least some of the feature extraction units in the plurality of feature extraction units; and detect a target text region in each of the chip images from the chip image based on the target fusion features obtained by the multi-level fusion;

[0014] The recognition module is used to perform text recognition on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0015] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0016] Acquire a chip image; the chip image includes at least one chip image, each of which is an image of the chip surface where the chip identifier is located;

[0017] Based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence;

[0018] Performing multi-level fusion according to the features extracted by at least some of the feature extraction units among the multiple feature extraction units;

[0019] Based on the target fusion features obtained by the multi-level fusion, detecting the target text area in each chip image from the chip image;

[0020] Text recognition is performed on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0022] Acquire a chip image; the chip image includes at least one chip image, each of which is an image of the chip surface where the chip identifier is located;

[0023] Based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence;

[0024] Performing multi-level fusion according to the features extracted by at least some of the feature extraction units among the multiple feature extraction units;

[0025] Based on the target fusion features obtained by the multi-level fusion, detecting the target text area in each chip image from the chip image;

[0026] Text recognition is performed on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0027] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0028] Acquire a chip image; the chip image includes at least one chip image, each of which is an image of the chip surface where the chip identifier is located;

[0029] Based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence;

[0030] Performing multi-level fusion according to the features extracted by at least some of the feature extraction units among the multiple feature extraction units;

[0031] Based on the target fusion features obtained by the multi-level fusion, detecting the target text area in each chip image from the chip image;

[0032] Text recognition is performed on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0033] The chip identifier recognition method, device, computer equipment, medium and product described above obtain a chip image including at least one chip image, and based on the chip image, perform feature extraction in sequence through a plurality of feature extraction units connected in sequence, and perform multi-level fusion according to the features extracted by at least some of the feature extraction units in the plurality of feature extraction units, so that the features of the text area in each chip image in the chip image can be more accurately extracted, so that based on the target fusion features obtained by the multi-level fusion, the target text area in each chip image can be more accurately detected, and then text recognition is performed on the target text area of ​​each chip image, so that the identifier of the chip corresponding to each chip image can be identified. In this way, the identifier of at least one chip can be automatically identified, and the recognition efficiency of the chip identifier is improved compared with the manual visual inspection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] FIG1 is a diagram showing an application environment of a chip identifier recognition method according to an embodiment;

[0036] FIG2 is a schematic diagram of a flow chart of a chip identifier recognition method in one embodiment;

[0037] FIG3 is a schematic diagram of a text detection network structure in one embodiment;

[0038] FIG4 is a schematic diagram of a target text area detection process in one embodiment;

[0039] FIG5 is a block diagram of a chip identifier recognition device in one embodiment;

[0040] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The chip identifier recognition method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1. The image acquisition device 102 can communicate with the computer device 104. The image acquisition device 102 can acquire chip images, and the computer device 104 can obtain the chip images acquired by the image acquisition device 102. Based on the chip images, multiple feature extraction units connected in sequence perform feature extraction in sequence, and multi-level fusion is performed according to the features extracted by at least some of the feature extraction units in the multiple feature extraction units. Based on the target fusion features obtained by the multi-level fusion, the target text area in each chip image is detected from the chip image, and the target text area of ​​each chip image is subjected to text recognition to obtain the identifier of the chip corresponding to each chip image. The image acquisition device 102 can be a camera or a camera group formed by multiple cameras. The computer device 104 can be a terminal, a server, or a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal can be a personal computer, a laptop, a smart phone, a tablet computer, an industrial computer or other. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0043] In an exemplary embodiment, as shown in FIG2 , a chip identifier recognition method is provided, which is described by taking the method applied to the computer device 104 in FIG1 as an example, and includes the following steps 202 to 210. Among them:

[0044] Step 202, obtaining a chip image; the chip image includes at least one chip image, each chip image is an image of the chip surface where the chip identifier is located.

[0045] Among them, a chip is a microelectronic circuit that integrates tiny electronic components on a single semiconductor substrate. Chips can include optical chips, electrical chips, or various other chips. Optical chips can specifically include VCSEL (Vertical-Cavity Surface-Emitting Laser) chips. VCSEL chips have the advantages of good laser output characteristics and can be formed into arrays, and can be used in fields such as automotive lidar. VCSEL chips are also small in size and have many specifications, so it is difficult to identify the identifier of the VCSEL chip.

[0046] A chip image is an image obtained by photographing the chip surface of at least one chip. The at least one chip may be one chip or two or more chips. The chip surface is the surface where the chip identifier is located. A chip image is an image area in a chip image, which represents the chip surface of a chip. The chip identifier is information used to distinguish different chips. The chip identifier may also be called the chip ID (Identity Document). The chip identifier may include the chip model, serial number or part number (Part No.). The identifier may specifically include characters such as numbers, letters, and symbols.

[0047] Exemplarily, the computer device may acquire the chip image acquired in real time by the image acquisition device. The computer device may communicate with the image acquisition device, and when the image acquisition device acquires a chip image, the image acquisition device may transmit the acquired chip image to the computer device. It is understood that in another embodiment, the image acquisition device may also store the acquired chip image in a local or preconfigured storage device, and the computer device may periodically acquire the stored chip image.

[0048] In one embodiment, obtaining a chip image and performing chip identifier identification can be specifically implemented by a chip identification system, which may include a work platform, a motion axis, a visual recognition system, a position scanning system, and control software. Among them, the work platform is used to place multiple chips to be identified, and the multiple chips may be chips of different specifications. The control software can be run in a terminal, and the control software can be used to control the operation of the position scanning system or the visual recognition system. The position scanning system can be implemented by a position scanning device and a computer device. The position scanning device can be a camera, or a camera group formed by multiple cameras. The visual recognition system can be implemented by an image acquisition device and a computer device. The motion axis can be used to rotate the camera in the position scanning system or the visual recognition system for scanning or shooting.

[0049] For example, in the process of identifying the chip, the position scanning system can be controlled by the control software to identify the starting point coordinates in the work platform and the position of each chip to complete product positioning, and the visual recognition system can be controlled to adjust the camera focal length and magnification to obtain a clear chip image to implement step 202. The visual recognition system identifies the identifier in the chip image to implement steps 204 to 210, and then the position and identifier of each chip are entered accordingly.

[0050] Step 204 , based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence.

[0051] Among them, the feature extraction unit is a unit based on the convolution layer. Feature extraction may include convolution processing. The feature extracted by the feature extraction unit is a feature map, which can specifically represent the text area where the identifier is located in the extracted chip image. Among the multiple feature extraction units connected in sequence, the number of convolution kernels used when performing convolution processing on the feature extraction units that are later in the connection order increases successively compared to the convolution kernels used when performing convolution processing on the feature extraction units that are earlier in the connection order. Among the features extracted by each of the multiple feature extraction units, the feature size decreases successively. The feature size is the width and height of the feature map.

[0052] Exemplarily, the computer device may perform feature extraction on an input image determined based on a chip image through a first feature extraction unit among a plurality of feature extraction units connected in sequence, and obtain features extracted by the first feature extraction unit; and from a second feature extraction unit among a plurality of feature extraction units connected in sequence, perform feature extraction on features extracted by a feature extraction unit preceding in the connection order through each feature extraction unit, and obtain features extracted by the feature extraction unit. The input image may be a chip image or a preprocessed chip image.

[0053] Step 206: performing multi-level fusion based on the features extracted by at least some of the feature extraction units among the multiple feature extraction units.

[0054] The at least part of the feature extraction units is at least part of the multiple feature extraction units. The at least part of the feature extraction units can be part of the multiple feature extraction units or multiple feature extraction units.

[0055] Exemplarily, the computer device may perform ASPP (Atrous Spatial Pyramid Pooling) processing on the features extracted by the last feature extraction unit among multiple feature extraction units to obtain enhanced features, and after up-pooling the enhanced features, perform multi-level fusion with the features extracted by other feature extraction units in at least some of the feature extraction units to obtain target fused features.

[0056] Among them, other feature extraction units are feature extraction units other than the last feature extraction unit in at least some of the feature extraction units. ASPP processing can expand the receptive field. The receptive field is the size of the area where the pixels on the feature map output by the feature extraction unit (i.e., the features extracted by the feature extraction unit) are mapped back to the input image. The upper pooling process is the inverse operation of the maximum pooling.

[0057] In one embodiment, the plurality of feature extraction units connected in sequence may include five feature extraction units, at least some of the feature extraction units may be the four feature extraction units after the first feature extraction unit in the plurality of feature extraction units, and the features extracted by at least some of the feature extraction units are, in order of connection, the first feature, the second feature, the third feature, and the fourth feature. In this embodiment, the computer device may fuse the fourth feature and the third feature to obtain the first fused feature, and fuse the first fused feature, the first feature, and the second feature to obtain the target fused feature.

[0058] In one embodiment, the computer device may perform ASPP processing on each of the fourth feature and the third feature respectively to obtain enhanced features corresponding to the fourth feature and the third feature respectively; after performing pooling processing on the enhanced features corresponding to the fourth feature, they are concatenated with the enhanced features corresponding to the third feature to obtain the first fused feature.

[0059] Step 208 , based on the target fusion features obtained by multi-level fusion, the target text area in each chip image is detected from the chip image.

[0060] The target text area is the text area where the identifier of the chip in the detected chip image is located. It can be understood that the target text area can be one or more. For example, in the chip surface, when the identifier is composed of a line of characters, the target text area can be one, and when the identifier is composed of multiple lines of characters, the target text area can be multiple.

[0061] Exemplarily, the computer device may input the target fusion features into the output layer of a pre-trained text detection network for processing to obtain multiple candidate text regions, and screen out the target text region from the multiple candidate text regions through a non-maximum suppression (NMS) operation.

[0062] The text detection network can be an EAST network (Efficient and Accurate Scene Textdetector, an efficient and accurate natural scene text detection network proposed by Megvii Technology), a YOLO series network (YouOnly Look Once, a target detection network based on deep learning), or others. The output layer can be the Output Layer in the EAST network or the Head layer (detection head) in the YOLO series network.

[0063] The output layer of the text detection network may output the respective positions and respective confidences of multiple candidate text regions, so as to represent the multiple candidate text regions obtained by the output layer of the text detection network. The confidence of a candidate text region characterizes the probability that the candidate text region contains the text in the identifier, and the accuracy of the candidate text region in identifying the location of the text in the identifier. The candidate text region may be a rectangular frame or an irregular quadrilateral. In the case where the candidate text region is a rectangular frame, the position of the candidate text region may include the size of the candidate text region, the center coordinate point of the candidate text region, and the rotation degree of the candidate text region. In the case where the candidate text region is an irregular quadrilateral, the position of the candidate text region may include the coordinates of the four vertices of the candidate text region.

[0064] The non-maximum suppression operation may be a standard non-maximum suppression operation. The standard non-maximum suppression operation may specifically include the following steps: obtaining a preset confidence threshold, arranging multiple candidate text regions in descending order according to confidence, obtaining a candidate region list, selecting the candidate text region with the highest confidence from the candidate region list as the current text region and adding it to the output list, deleting the current text region from the candidate region list, calculating the IoU value (Intersection over Union) between the current text region and all candidate text regions in the candidate region list, deleting candidate text regions greater than the confidence threshold, repeating the above process until the candidate region list is empty, and determining the candidate text region in the obtained output list as the target text region.

[0065] The non-maximum suppression operation can also be a locality-aware non-maximum suppression (Locality-Aware NMS) operation. The locality-aware non-maximum suppression operation specifically treats each line as a spatial region and performs a standard non-maximum suppression operation on the candidate text regions in the same spatial region, which can reduce the amount of calculation.

[0066] Step 210 , performing text recognition on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

[0067] Exemplarily, a pre-trained OCR (Optical Character Recognition) model is used to perform text recognition on a target text area of ​​each chip image to obtain an identifier of a chip corresponding to each chip image.

[0068] The OCR model may be Tesseract (an open source OCR engine maintained by Google), EasyOCR (an open source OCR library), or others. The OCR model may be trained using sample chip images and corresponding labels, wherein the sample chip image includes a chip image of at least one sample chip, and the label corresponding to the sample chip image may include a sample text area where an identifier of the sample chip is located in the sample chip image, and text in the sample text area.

[0069] In the above-mentioned chip identifier recognition method, a chip image including at least one chip image is obtained, and based on the chip image, feature extraction is performed in sequence by multiple feature extraction units connected in sequence. Multi-level fusion is performed according to the features extracted by at least some of the multiple feature extraction units. The features of the text area in each chip image in the chip image can be extracted more accurately, so that based on the target fusion features obtained by multi-level fusion, the target text area in each chip image can be detected more accurately, and then text recognition is performed on the target text area of ​​each chip image, and the identifier of the chip corresponding to each chip image can be identified. In this way, the identifier of at least one chip can be automatically identified, and the recognition efficiency of the chip identifier is improved compared with the manual visual inspection method.

[0070] In an exemplary embodiment, the chip image is a color image. Before step 204, the chip identifier recognition method further includes: after compressing the chip image, adjusting the image size to a preset size and adjusting the image format to a preset format to obtain a compressed image; performing image equalization on the compressed image to obtain a balanced image; performing filtering on the balanced image to obtain a filtered image; converting the filtered image into a grayscale image, binarizing the grayscale image, and performing rotation correction to obtain a preprocessed chip image.

[0071] Among them, the compression process is to compress the size of the chip image. The compression process can be a compression process using the JPEG (Joint Photographic Experts Group, a static image compression standard developed by the Joint Photographic Experts Group of ISO and IEC) algorithm. Since the pixel of the chip image camera is more than 10 million, the size of the chip image captured is usually tens of MB (megabytes), which occupies a large memory space of the computer device during processing, reduces the processing efficiency of the computer device, increases the processing time, and does not meet the real-time processing requirements in the industry. At the same time, since the chip image may include chip images of multiple chips, and the identifier on the chip surface represented by the chip image is small, therefore, when compressing the chip image, the detailed information of the text area where the identifier is located must be retained. Using the JPEG algorithm for compression processing can greatly retain the image detail information. The size of the compressed chip image may not exceed 512KB (kilobytes). The preset image size is a pre-set image size, for example, it can be 256*256. The preset image format is a pre-set image format. It can be understood that the image format after being processed by the JPEG algorithm is ".jpg" or ".jpeg", and the preset image format can be one of ".jpg" or ".jpeg".

[0072] Image equalization is to perform histogram equalization on the image. Image equalization can improve the contrast between the identifier and the background on the chip image, and can improve the accuracy of subsequent recognition of the text area and the text in the text area. For VCSEL chips, since VCSEL chips have a variety of specifications, extremely small sizes, and a variety of packaging methods, and during the shooting process, the image obtained often has uneven brightness or color due to the uneven placement of the chip or the influence of the light source. Through image equalization, the contrast of the image is increased, the image is clearer, and the difficulty of subsequent text recognition is reduced.

[0073] Filtering is the process of removing noise from an image. Since there is noise when acquiring image signals through the sensor in the camera, and there are also various noises in the actual environment, the captured image needs to be filtered. Since the Gaussian filter algorithm smoothes the edges while removing noise, the resulting image is blurred. Therefore, the filtering process can use a bilateral filtering algorithm, which can better retain edge information by introducing pixel value weight terms.

[0074] A grayscale image is a single-channel image with 256 grayscale levels. Binarization is the process of converting an image into a binary image by threshold segmentation. Binarization can be implemented using the OSTU algorithm (Otsu method - maximum inter-class variance method). Rotation correction is the process of rotating an image to bring it to a standard viewing angle.

[0075] In this embodiment, compression processing, image equalization processing, filtering processing, and improving the quality of the chip image can improve the efficiency or accuracy of subsequent text area detection and text recognition. After conversion to a grayscale image, binarization processing is performed to remove invalid information in the image, which can improve the efficiency of subsequent text area detection and text recognition.

[0076] In one embodiment, the computer device may use the JPEG algorithm to compress the chip image, adjust the image size to a preset image size, and adjust the image format to a preset image format to obtain a compressed image; perform image equalization on the compressed image to obtain an equalized image; use a bilateral filtering algorithm to filter the equalized image to obtain a filtered image, convert the filtered image into a grayscale image, use the OSTU algorithm to binarize it to obtain a binary image, and perform rotation correction on the binary image to obtain a preprocessed chip image.

[0077] In an exemplary embodiment, the above-mentioned step of performing image equalization processing on the compressed image to obtain a balanced image includes: obtaining an R channel image, a G channel image and a B channel image of the compressed image in an RGB space; performing histogram equalization processing on the R channel image, the G channel image and the B channel image, respectively, to obtain color balanced images corresponding to the R channel image, the G channel image and the B channel image; converting a brightness channel image, a saturation channel image and a hue channel image in an HSV space according to the color balanced images corresponding to the R channel image, the G channel image and the B channel image; performing histogram equalization processing on the brightness channel image to obtain a brightness channel balanced image; converting a balanced image in an RGB space according to the brightness channel balanced image, the saturation channel image and the hue channel image.

[0078] Among them, RGB space is a color space that describes colors based on the superposition of the three basic colors of red, green, and blue to varying degrees. The R channel image is the red channel image. The G channel image is the green channel image. The B channel image is the blue channel image. Histogram equalization is to redistribute pixel values ​​of a single channel image so that the grayscale distribution of the single channel image after the redistribution of pixel values ​​is balanced. HSV space is a color space created based on the intuitive characteristics of color. In the HSV space, there are three channels: hue (Hue, H) channel, saturation (Saturation, S) channel, and value (Value, V) channel. The brightness channel image and the brightness channel balanced image are images under the brightness channel, the saturation channel image is an image under the saturation channel, and the hue channel image is an image under the hue channel.

[0079] In this embodiment, the R channel image, G channel image and B channel image of the compressed image in the RGB space are respectively subjected to histogram equalization processing, which can preliminarily complete the image equalization processing and preliminarily enhance the contrast between the text and the background. Then, the color balanced images of the three are respectively converted into the HSV space, and the single channel image of the brightness channel obtained after the conversion is subjected to histogram equalization processing, which can further enhance the brightness characteristics, thereby improving the quality of the chip image.

[0080] In an exemplary embodiment, the above-mentioned step of filtering the equalized image to obtain a filtered image includes: calling a bilateral filtering function based on a preset filtering window diameter, a preset pixel value range variance, a preset spatial value range variance, and a preset edge processing method; filtering the equalized image through the bilateral filtering function to obtain a filtered image.

[0081] Among them, the bilateral filtering function is a function that uses a bilateral filtering algorithm for filtering. In the bilateral filtering algorithm, the kernel function of the filter is generated by two kernel functions, the spatial domain kernel and the value range kernel. The value range kernel can also be called a pixel value weight term. The bilateral filtering function can be called in a preconfigured function library. The function library can be OpenCV (a cross-platform computer vision library). When calling the bilateral filtering function, the preset filter window diameter, the preset pixel value range variance, the preset spatial value range variance, and the preset edge processing method can be used as input parameters of the bilateral filtering function.

[0082] The preset filter window diameter is the preset filter window diameter. The filter window diameter is the size of the pixel area affected when the filter is applied to the image. The preset filter window diameter is 1, 2. The preset pixel value domain variance is the preset pixel value domain variance. The pixel value domain variance is used to measure the degree of change of pixel values ​​when bilateral filtering is performed. The preset pixel value domain variance is 80, 100. The preset spatial value domain variance is the preset spatial value domain variance. The spatial value domain variance is used to measure the degree of change of spatial position when bilateral filtering is performed. The preset spatial value domain variance is 30, 40. The preset edge processing method is the preset edge processing method. The preset edge processing methods are such as filling the edge with a specified pixel value, filling the edge with an existing edge pixel value, or others.

[0083] In this embodiment, the equalized image is filtered using a bilateral filtering function, which can effectively remove noise in the equalized image and retain edge information in the equalized image, thereby improving the quality of the equalized image and facilitating subsequent text area extraction and identifier recognition.

[0084] In an exemplary embodiment, step 204 includes: based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence to obtain a first feature, a second feature, a third feature, and a fourth feature respectively extracted by four feature extraction units in the plurality of feature extraction units in the order of connection from first to last; wherein each feature extraction unit includes a convolution layer for convolution processing, and the number of convolution kernels in the convolution layer included in each of the plurality of feature extraction units increases in sequence according to the order of connection of the plurality of feature extraction units; the feature sizes of the first feature, the second feature, the third feature, and the fourth feature decrease in sequence.

[0085] Among them, the feature extraction unit performs feature extraction based on convolution processing. The multiple feature extraction units may include only four feature extraction units, or may include more than four feature extraction units. For example, the multiple feature extraction units may include five feature extraction units, wherein the last four feature extraction units sequentially output the first feature, the second feature, the third feature and the fourth feature, and each of the four feature extraction units outputs one feature. The number of convolution kernels in the convolution layer included in each of the multiple feature extraction units increases in sequence according to the connection order of the multiple feature extraction units. For example, according to the connection order, the multiple feature extraction units are respectively recorded as feature extraction unit 1, feature extraction unit 2, and feature extraction unit 3. Then, the number of convolution kernels in the convolution layer included in feature extraction unit 2 is more than that in feature extraction unit 1, and the number of convolution kernels in the convolution layer included in feature extraction unit 3 is more than that in feature extraction unit 2.

[0086] In this embodiment, four feature extraction units among the multiple feature extraction units respectively extract the first feature, the second feature, the third feature, and the fourth feature, creating conditions for subsequent multi-level fusion. Moreover, since the number of convolution kernels in the convolutional layers included in each of the multiple feature extraction units increases in sequence according to the order of connection, features of different richness can be extracted, and the feature sizes of the first feature, the second feature, the third feature, and the fourth feature decrease in sequence, and features of different sizes can be extracted. In this way, when multi-level fusion is performed subsequently, more accurate target fusion features can be obtained.

[0087] In an exemplary embodiment, the four feature extraction units include two first feature extraction units connected in sequence, and the two first feature extraction units extract the first feature and the second feature in sequence according to the connection order; wherein, in the process of feature extraction by each first feature extraction unit, convolution processing is performed on the features input to the first feature extraction unit, and the features after the convolution processing are inverted, and based on the features obtained by splicing the features after the convolution processing and the features after the inversion operation, the features extracted by the first feature extraction unit are output.

[0088] Wherein, before the two first feature extraction units connected in sequence, the plurality of feature extraction units may further include an initial feature extraction unit having the same structure as the first feature extraction unit, and a pooling layer after the initial feature extraction unit. The initial feature extraction unit is used to extract features from the input image. The input image may be a chip image or a preprocessed chip image.

[0089] The pooling layer is used to perform pooling processing on the features extracted by the initial feature extraction unit. Pooling processing is used to reduce the feature size. The pooling processing can be average pooling processing, maximum pooling processing or other processing. The features output by the pooling layer are used to input into the first feature extraction unit in the connection order of the two first feature extraction units for feature extraction.

[0090] Among the two first feature extraction units connected in sequence, the first feature extraction unit that is connected first in the order of connection extracts the first feature. The first feature is input into the first feature extraction unit that is connected later in the order of connection for feature extraction to obtain the second feature.

[0091] Negation is the process of negating the features after convolution. The features after the inverse convolution and the features after the negation operation are concatenated. Compared with the features after the convolution, the number of channels of the features is doubled. In other words, during the convolution process, fewer convolution kernels can be used, and features with more channels can be obtained through the negation operation and concatenation, which effectively improves the efficiency of feature extraction.

[0092] In this embodiment, the first feature extraction unit splices the features after convolution processing and the features after inversion operation during the feature extraction process. The number of feature channels after the splicing is doubled compared to the feature channels after the convolution processing, which can effectively reduce the number of convolution kernels in the convolution processing and improve the feature extraction efficiency.

[0093] In one embodiment, during the feature extraction process performed by each first feature extraction unit, a convolution process with a convolution kernel size of 1*1 is performed on the feature input to the first feature extraction unit, and then a convolution process with a convolution kernel size of 3*3 is performed to obtain the feature after the convolution process, and an inversion operation is performed on the feature after the convolution process, and the feature after the convolution process and the feature after the inversion operation are spliced ​​(called Concatenation or Concat) to obtain the spliced ​​feature, and batch normalization is performed on the spliced ​​feature, and then activation processing is performed, and convolution processing with a convolution kernel size of 1*1 is performed on the feature after the activation process to obtain the intermediate feature extracted by the first feature extraction unit, and the intermediate feature is added to the feature input to the first feature extraction unit to obtain the feature output by the first feature extraction unit, that is, the feature extracted by the first feature extraction unit.

[0094] Among them, batch normalization is used to standardize the features. In batch normalization, Scale operation and Shift operation can be performed to solve the problem of internal covariate shift during training. Activation processing can be implemented using an activation function. The activation function can be a ReLU (RectifiedLinearUnit, linear rectification) function. In the process of feature extraction through the initial feature extraction unit, unlike the feature extraction by the first feature extraction unit, the input of the initial feature extraction unit is a chip image or a preprocessed chip image, and the convolution kernel size used by the initial feature extraction unit can be 7*7, and the other parts can be the same as the processing performed by the first feature extraction unit for feature extraction.

[0095] In an exemplary embodiment, the four feature extraction units also include two second feature extraction units connected in sequence after the two first feature extraction units, and the two second feature extraction units extract the third feature and the fourth feature in sequence in the order of connection; wherein, in the process of feature extraction by each second feature extraction unit, different multiple convolution processes are performed on the features input into the second feature extraction unit, so that the sizes of the features extracted by the multiple convolution processes are different, and the features extracted by the second feature extraction unit are output based on the features after splicing the features extracted by the multiple convolution processes.

[0096] The second feature is input into the first second feature extraction unit in the connection order of the two second feature extraction units connected in sequence, and feature extraction is performed to obtain the third feature. The third feature is input into the second feature extraction unit that is later in the connection order, and feature extraction is performed to obtain the fourth feature.

[0097] In different multiple convolution processing, the first convolution processing may be a convolution processing with a convolution kernel size of 1*1, the second convolution processing may be a combination of two steps of first performing a convolution processing with a convolution kernel size of 1*1 and then performing a convolution processing with a convolution kernel size of 3*3, and the third convolution processing may be a combination of three steps of first performing a convolution processing with a convolution kernel size of 1*1 and then performing a convolution processing with a convolution kernel size of 3*3 and finally performing a convolution processing with a convolution kernel size of 3*3. Different multiple convolution processing may also include a fourth convolution processing, which may be a combination of two steps of first performing a pooling processing with a size of 3*3 and then performing a convolution processing with a convolution kernel size of 1*1.

[0098] In this embodiment, after the first feature extraction unit, in the process of feature extraction performed by the second feature extraction unit, the sizes of features extracted by the multiple convolution processes are different. The features extracted by the multiple convolution processes are spliced ​​together to capture the text areas corresponding to identifiers of different sizes in the chip image. In this way, the features representing the text areas in the chip image can be accurately extracted, thereby improving the accuracy of text area detection.

[0099] In one embodiment, during the feature extraction process performed by each second feature extraction unit, different multiple convolution processes are performed on the features input into the second feature extraction unit, so that the sizes of the features extracted by the multiple convolution processes are different, the features extracted by the multiple convolution processes are spliced ​​to obtain the spliced ​​features, and the spliced ​​features are convoluted with a convolution kernel size of 1*1 to obtain the intermediate features extracted by the second feature extraction unit, and the intermediate features are added to the features input into the second feature extraction unit to obtain the features output by the second feature extraction unit, i.e., the features extracted by the second feature extraction unit.

[0100] In an exemplary embodiment, step 206 includes: fusing the fourth feature and the third feature to obtain a first fused feature; performing convolution processing and pooling processing on the first fused feature in sequence, and then splicing it with the second feature to obtain a second fused feature; performing convolution processing and pooling processing on the second fused feature in sequence, and then splicing it with the first feature to obtain a third fused feature; performing convolution processing on the third fused feature to obtain a multi-level fused target fused feature.

[0101] In this embodiment, the first feature, the second feature, the third feature and the fourth feature, the feature levels are from shallow to deep, the feature richness increases successively, and the receptive field decreases successively. The fourth feature and the third feature are fused to obtain the first fused feature, and then the first fused feature and the second feature are fused to obtain the second fused feature, and the second fused feature and the first feature are spliced ​​to obtain the third fused feature. In this way, the deep features and the shallow features can be fused in sequence, and the obtained target fused features take into account both feature richness and receptive field, and can more accurately express the text area in the chip image.

[0102] In one embodiment, the step of fusing the fourth feature with the third feature to obtain the first fused feature may include: the computer device may perform pooling processing on the fourth feature and then concatenate it with the third feature to obtain the first fused feature.

[0103] In one embodiment, the step of fusing the fourth feature and the third feature to obtain the first fused feature may include: performing atrous convolution processing with a preset expansion rate on the fourth feature and the third feature, respectively, to obtain intermediate features corresponding to the targeted features; performing convolution processing on the fourth feature and the third feature, respectively, to obtain convolution features corresponding to the targeted features; fusing the intermediate features and convolution features corresponding to the targeted features to obtain enhanced features corresponding to the targeted features; and performing pooling processing on the enhanced features corresponding to the fourth feature, and then concatenating them with the enhanced features corresponding to the third feature to obtain the first fused feature.

[0104] Among them, in the dilated convolution processing, the convolution kernel used is the dilated convolution kernel obtained by introducing zero elements into the standard convolution kernel, which can increase the receptive field of the convolution kernel. The dilated convolution processing can also be called the dilated convolution processing or the dilated convolution processing. The standard convolution kernel is the convolution kernel before dilation.

[0105] The dilation rate can be used to measure the size relationship between the dilated convolution kernel and the standard convolution kernel, as well as the distance between adjacent elements of the standard convolution kernel in the dilated convolution kernel. The preset dilation rate is a pre-set dilation rate. The preset dilation rate is, for example, 2, 4, or other. The preset dilation rate can be selected by user or obtained through training. During training, the training goal can be to achieve a preset value for the accuracy of the target text area in the chip image and to have a recognition time less than the preset time.

[0106] In this embodiment, a dilated convolution process with a preset expansion rate is performed on the fourth feature and the third feature respectively, so that the receptive field can be expanded without increasing the number of convolution processes, thereby capturing a wider range of information in the chip image. The intermediate features and convolution features corresponding to the targeted features are fused, and the fused features can include features of different sizes, thereby improving the feature richness and thus the expressiveness of the features, so that the first fused features obtained subsequently can more accurately express the target text area in the chip image.

[0107] In one embodiment, the computer device may perform dilated convolution processing with different preset expansion rates on the fourth feature, obtain intermediate features corresponding to the fourth feature at each preset expansion rate, and fuse the intermediate features corresponding to the fourth feature at different preset expansion rates to obtain enhanced features corresponding to the fourth feature. It can be understood that the acquisition process of the enhanced feature corresponding to the third feature is similar to that of the fourth feature, and will not be repeated here. The dilated convolution processing with different preset expansion rates may be, for example, three dilated convolution processings, each of which uses a different preset expansion rate.

[0108] In one embodiment, the computer device may fuse the intermediate features corresponding to the fourth feature at different preset expansion rates to obtain an enhanced feature corresponding to the fourth feature.

[0109] In one embodiment, the computer device may perform convolution processing on each of the fourth feature and the third feature to obtain a convolution feature corresponding to the targeted feature; perform hole convolution processing with different preset expansion rates on the targeted features to obtain intermediate features corresponding to the targeted features at each preset expansion rate; perform pooling processing, convolution processing and upsampling processing on the targeted features in sequence to obtain an upsampled feature corresponding to the targeted feature; splice the convolution feature corresponding to the targeted feature, the intermediate features corresponding to the targeted feature at different preset expansion rates, and the upsampled features corresponding to the targeted feature to obtain a spliced ​​feature; perform convolution processing and upsampling processing on the spliced ​​feature in sequence to obtain an enhanced feature corresponding to the targeted feature. In this embodiment, the convolution kernel size used in the convolution processing may be 1*1, and the upsampling processing may use a bilinear interpolation algorithm.

[0110] In an application scenario, the chip image may be a VCSEL chip image, the target text area of ​​each chip image in the VCSEL chip image may be obtained by inputting the chip image after preprocessing the VCSEL chip image into a pretrained text detection network for detection, and the identifier in the VCSEL chip image may be obtained by performing text recognition through a pretrained OCR model. When training the text detection network and the OCR model, a data set including a training set and a test set may be constructed for training, and the ratio of the training set to the test set may be 4:1. The data set may include multiple sample chip images, each of which may include chip images of multiple VCSEL chips, and the VCSEL chip may be a VCSEL chip used in the radar field or a VCSEL chip used in the data communication field. The data set may also record the specifications and material status of the VCSEL chip. During the training process, the trained network may be evaluated from three indicators: accuracy, recall, and F1-measure (F1 indicator, which combines accuracy and recall).

[0111] Referring to the schematic diagram of the text detection network structure shown in FIG3 , the text detection network can be called a fully convolutional network, and the text detection network can include a feature extraction subnetwork, a feature fusion subnetwork, and an output layer. Among them, the feature extraction subnetwork may include a plurality of feature extraction units connected in sequence, and the feature extraction subnetwork is used for chip images, and feature extraction is performed in sequence through a plurality of feature extraction units connected in sequence. The plurality of feature extraction units connected in sequence are, in sequence, an initial feature extraction unit, two first feature extraction units, and two second feature extraction units. The feature fusion subnetwork is used to perform multi-level fusion based on the features extracted by some feature extraction units in the plurality of feature extraction units; the output layer is used to detect the target text area in each chip image from the chip image based on the target fusion features obtained by multi-level fusion.

[0112] Based on FIG. 3 and the target text area detection process diagram shown in FIG. 4 , the chip identifier recognition method may specifically include the following steps.

[0113] The computer device can obtain the chip image, compress the chip image, adjust the image size to a preset size, adjust the image format to a preset format, and obtain a compressed image. The R channel image, the G channel image, and the B channel image in the RGB control of the compressed image are respectively subjected to histogram equalization to obtain color balanced images of the R channel image, the G channel image, and the B channel image; the color balanced images of the R channel image, the G channel image, and the B channel image are respectively converted into brightness channel images, saturation channel images, and hue channel images in the HSV space; the brightness channel image is subjected to histogram equalization to obtain a brightness channel balanced image; according to the brightness channel balanced image, the saturation channel image, and the hue channel image, a balanced image in the RGB space is converted; the balanced image is filtered to obtain a filtered image; after converting the filtered image into a grayscale image, the grayscale image is binarized and then rotationally corrected to obtain a preprocessed chip image.

[0114] The computer device can perform feature extraction on the preprocessed chip image through the initial feature extraction unit to obtain the features extracted by the initial feature extraction unit. The initial feature extraction unit can use a convolution kernel with a size of 7*7, and the number of channels of the preprocessed chip image is 3. After the initial feature extraction unit performs feature extraction, the number of channels of the obtained features can be increased to 16, and the size of the features is 1 / 2 of the size of the preprocessed chip image.

[0115] The computer device may extract features extracted by the initial feature extraction unit through the first feature extraction unit (Conv stage 1) of the two first feature extraction units that precedes the first feature extraction unit in the connection order, to obtain a first feature. The number of channels of the first feature may be 64, and the size of the first feature is 1 / 2 of the size of the feature extracted by the initial feature extraction unit.

[0116] The computer device may extract the first feature through the first feature extraction unit (Conv stage 2) that is later in the connection order among the two first feature extraction units to obtain the second feature. The number of channels of the second feature may be 128, and the size of the second feature may be 1 / 2 of the size of the first feature.

[0117] The computer device may extract the second feature through the second feature extraction unit (Conv stage 3) that is earlier in the connection order among the two second feature extraction units to obtain the third feature. The number of channels of the third feature may be 256, and the size of the third feature may be 1 / 2 of the size of the second feature.

[0118] The computer device may extract the third feature through the second feature extraction unit (Conv stage 4) that is later in the connection order among the two second feature extraction units to obtain a fourth feature. The number of channels of the fourth feature may be 384, and the size of the fourth feature may be 1 / 2 of the size of the third feature.

[0119] The computer device can perform ASPP processing on the third feature. During the ASPP processing, the third feature is subjected to dilated convolution processing with different preset expansion rates to obtain an intermediate feature corresponding to the third feature at each preset expansion rate. The intermediate features corresponding to the third feature at different preset expansion rates are respectively fused to obtain an enhanced feature corresponding to the third feature.

[0120] The computer device can perform ASPP processing on the fourth feature. During the ASPP processing, the fourth feature is subjected to dilated convolution processing with different preset expansion rates to obtain an intermediate feature corresponding to the fourth feature at each preset expansion rate. The intermediate features corresponding to the fourth feature at different preset expansion rates are respectively fused to obtain an enhanced feature corresponding to the fourth feature.

[0121] The computer device may perform unpooling on the enhanced feature corresponding to the fourth feature, and then concatenate it with the enhanced feature corresponding to the third feature to obtain a first fused feature. The unpooling process may increase the size of the feature by 2 times.

[0122] The computer device can perform convolution processing with a convolution kernel size of 1*1 and a number of convolution kernels of 128, and convolution processing with a convolution kernel size of 3*3 and a number of convolution kernels of 128 on the first fusion feature, and then perform pooling processing, and then splice it with the second feature to obtain the second fusion feature.

[0123] The computer device can perform convolution processing with a convolution kernel size of 1*1 and 64 convolution kernels, and convolution processing with a convolution kernel size of 3*3 and 64 convolution kernels on the second fusion feature, and then perform pooling processing, and then splice it with the first feature to obtain the third fusion feature.

[0124] The computer device can perform convolution processing with a convolution kernel size of 1*1 and a number of convolution kernels of 32, convolution processing with a convolution kernel size of 3*3 and a number of convolution kernels of 32, and convolution processing with a convolution kernel size of 3*3 and a number of convolution kernels of 32 on the third fusion feature in sequence to obtain the target fusion feature.

[0125] The computer device can process the target fusion features through the output layer to obtain the respective positions (RBOX and / or QUAD) and respective confidences (Score map) of multiple candidate text regions, and screen out the target text region from multiple candidate text regions through non-maximum suppression operations. Wherein, in the case where the candidate text region is a rectangular box (RBOX), the position of the candidate text region may include the size of the candidate text region, the center coordinate point of the candidate text region (the size and the center coordinate point may be the four parameters of the Text boxes) and the rotation degree (Text rotationangle) of the candidate text region. In the case where the candidate text region is an irregular quadrilateral (QUAD), the position of the candidate text region may include the coordinates of the four vertices of the candidate text region.

[0126] The computer device can perform text recognition on the target text area of ​​each chip image to obtain the identifier of the chip corresponding to each chip image.

[0127] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0128] Based on the same inventive concept, the embodiment of the present application also provides a chip identifier recognition device for implementing the chip identifier recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more chip identifier recognition device embodiments provided below can refer to the limitations of the chip identifier recognition method above, and will not be repeated here.

[0129] In an exemplary embodiment, as shown in FIG5 , a chip identifier recognition device 500 is provided, comprising: an acquisition module 510, an area detection module 520 and an identification module 530, wherein:

[0130] The acquisition module 510 is used to acquire a chip image; the chip image includes at least one chip image, and each chip image is an image of the chip surface where the chip identifier is located.

[0131] The region detection module 520 is used to perform feature extraction in sequence based on the chip image through multiple feature extraction units connected in sequence; perform multi-level fusion based on the features extracted by at least some of the multiple feature extraction units; and detect the target text area in each chip image from the chip image based on the target fusion features obtained by the multi-level fusion.

[0132] The recognition module 530 is used to perform text recognition on the target text area of ​​each chip image to obtain the identifier of the chip corresponding to each chip image.

[0133] In an exemplary embodiment, the chip image is a color image, and the region detection module 520 is also used to compress the chip image, adjust the image size to a preset size, and adjust the image format to a preset format to obtain a compressed image; perform image equalization on the compressed image to obtain a balanced image; perform filtering on the balanced image to obtain a filtered image; convert the filtered image into a grayscale image, binarize the grayscale image, and then perform rotation correction to obtain a preprocessed chip image.

[0134] In an exemplary embodiment, the region detection module 520 is also used to obtain the R channel image, G channel image and B channel image of the compressed image in the RGB space; perform histogram equalization processing on the R channel image, G channel image and B channel image, respectively, to obtain the color balanced images corresponding to the R channel image, G channel image and B channel image; convert the brightness channel image, saturation channel image and hue channel image in the HSV space according to the color balanced images corresponding to the R channel image, G channel image and B channel image; perform histogram equalization processing on the brightness channel image to obtain a brightness channel balanced image; convert the brightness channel balanced image, saturation channel image and hue channel image to obtain a balanced image in the RGB space.

[0135] In an exemplary embodiment, the region detection module 520 is also used to call a bilateral filtering function based on a preset filter window diameter, a preset pixel value range variance, a preset spatial value range variance, and a preset edge processing method; the equalized image is filtered through the bilateral filtering function to obtain a filtered image.

[0136] In an exemplary embodiment, the region detection module 520 is also used to perform feature extraction in sequence based on the chip image through multiple feature extraction units connected in sequence, and obtain the first feature, second feature, third feature and fourth feature extracted by four feature extraction units in the multiple feature extraction units in the order of connection from first to last; wherein each feature extraction unit includes a convolution layer for convolution processing, and the number of convolution kernels in the convolution layer included in each of the multiple feature extraction units increases in sequence according to the order of connection of the multiple feature extraction units; the feature sizes of the first feature, the second feature, the third feature and the fourth feature decrease in sequence.

[0137] In an exemplary embodiment, the four feature extraction units include two first feature extraction units connected in sequence, and the two first feature extraction units extract the first feature and the second feature in sequence according to the connection order; wherein, in the process of feature extraction by each first feature extraction unit, convolution processing is performed on the features input to the first feature extraction unit, and the features after the convolution processing are inverted, and based on the features obtained by splicing the features after the convolution processing and the features after the inversion operation, the features extracted by the first feature extraction unit are output.

[0138] In an exemplary embodiment, the four feature extraction units also include two second feature extraction units connected in sequence after the two first feature extraction units, and the two second feature extraction units extract the third feature and the fourth feature in sequence in the order of connection; wherein, in the process of feature extraction by each second feature extraction unit, different multiple convolution processes are performed on the features input into the second feature extraction unit, so that the sizes of the features extracted by the multiple convolution processes are different, and the features extracted by the second feature extraction unit are output based on the features after splicing the features extracted by the multiple convolution processes.

[0139] In an exemplary embodiment, the region detection module 520 is also used to fuse the fourth feature and the third feature to obtain a first fused feature; the first fused feature is sequentially convolved and pooled, and then concatenated with the second feature to obtain a second fused feature; the second fused feature is sequentially convolved and pooled, and then concatenated with the first feature to obtain a third fused feature; the third fused feature is convolved to obtain a multi-level fused target fused feature.

[0140] In an exemplary embodiment, the region detection module 520 is also used to perform hole convolution processing with a preset expansion rate on the fourth feature and the third feature, respectively, to obtain intermediate features corresponding to the targeted features; perform convolution processing on the fourth feature and the third feature, respectively, to obtain convolution features corresponding to the targeted features; fuse the intermediate features and convolution features corresponding to the targeted features to obtain enhanced features corresponding to the targeted features; perform up-pooling processing on the enhanced features corresponding to the fourth feature, and then splice them with the enhanced features corresponding to the third feature to obtain a first fused feature.

[0141] In an exemplary embodiment, the region detection module 520 is also used to input the target fusion features into the output layer of the pre-trained text detection network for processing to obtain multiple candidate text regions; and filter out the target text region from the multiple candidate text regions through a non-maximum suppression operation.

[0142] Each module in the above chip identifier recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0143] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a chip identifier recognition method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0144] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0147] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0148] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A chip identifier recognition method, characterized in that: The method comprises: Acquire a chip image; the chip image includes chip images of multiple chips of different specifications, each of which is an image of the chip surface where the chip identifier is located; Based on the chip image, feature extraction is performed in sequence by a plurality of feature extraction units connected in sequence, thereby obtaining a first feature and a second feature extracted in sequence by two first feature extraction units connected in sequence among the plurality of feature extraction units in the order of connection, and a third feature and a fourth feature extracted in sequence by two second feature extraction units connected in sequence after the two first feature extraction units in the order of connection; the number of convolution kernels in a convolution layer for convolution processing included in each of the plurality of feature extraction units increases in sequence according to the order of connection of the plurality of feature extraction units; A first fused feature is obtained by fusing the fourth feature and the third feature, and a target fused feature is obtained by fusing the first fused feature, the first feature and the second feature; the feature extracted by each first feature extraction unit is obtained by performing a convolution operation on the input of the first feature extraction unit and then performing an inversion operation, and is based on the feature obtained by the convolution operation and the feature obtained by the inversion operation; the feature extracted by each second feature extraction unit is obtained by performing a plurality of different convolution operations on the input of the second feature extraction unit and is based on the features extracted by the plurality of convolution operations; the sizes of the features extracted by the plurality of convolution operations are different; Based on the target fusion feature, detecting a target text area in each chip image from the chip image; Text recognition is performed on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

2. The method according to claim 1, characterized in that: The chip image is a color image. Before the step of sequentially extracting features based on the chip image by using a plurality of sequentially connected feature extraction units, the method further includes: After compressing the chip image, adjusting the image size to a preset size and the image format to a preset format to obtain a compressed image; Performing image equalization processing on the compressed image to obtain an equalized image; Performing filtering on the equalized image to obtain a filtered image; The filtered image is converted into a grayscale image, and after binarization processing is performed on the grayscale image, rotation correction is performed to obtain a preprocessed chip image.

3. The method according to claim 2, characterized in that The performing image equalization processing on the compressed image to obtain an equalized image includes: Obtain an R channel image, a G channel image, and a B channel image of the compressed image in an RGB space; Performing histogram equalization processing on the R channel image, the G channel image and the B channel image, respectively, to obtain color balanced images corresponding to the R channel image, the G channel image and the B channel image; According to the color balanced images corresponding to the R channel image, the G channel image and the B channel image, respectively, converting to obtain a brightness channel image, a saturation channel image and a hue channel image in the HSV space; Performing histogram equalization processing on the brightness channel image to obtain a brightness channel equalized image; A balanced image in RGB space is obtained by conversion according to the brightness channel balanced image, the saturation channel image and the hue channel image.

4. The method according to claim 2, characterized in that: The filtering process is performed on the equalized image to obtain a filtered image, including: Based on the preset filter window diameter, the preset pixel value range variance, the preset spatial value range variance, and the preset edge processing method, the bilateral filter function is called; The equalized image is filtered by the bilateral filtering function to obtain a filtered image.

5. The method according to claim 1, characterized in that: The first feature, the second feature, the third feature and the fourth feature have feature sizes that decrease in sequence.

6. The method according to claim 1, characterized in that In the process of feature extraction performed by each first feature extraction unit, convolution processing is performed on the features input to the first feature extraction unit, and an inversion operation is performed on the features after the convolution processing. Based on the features obtained by splicing the features after the convolution processing and the features after the inversion operation, the features extracted by the first feature extraction unit are output.

7. The method according to claim 1, characterized in that In the process of feature extraction performed by each second feature extraction unit, different multiple convolution processes are performed on the features input into the second feature extraction unit, so that the sizes of the features extracted by each of the multiple convolution processes are different. Based on the features after splicing the features extracted by each of the multiple convolution processes, the features extracted by the second feature extraction unit are output.

8. The method according to claim 1, characterized in that The step of fusing the first fusion feature, the first feature, and the second feature to obtain a target fusion feature includes: After performing convolution processing and pooling processing on the first fused feature in sequence, the first fused feature is concatenated with the second feature to obtain a second fused feature; After performing convolution processing and pooling processing on the second fused feature in sequence, the second fused feature is concatenated with the first feature to obtain a third fused feature; The third fusion feature is convolved to obtain a target fusion feature.

9. The method according to any one of claims 1 to 8, characterized in that: The fusing the fourth feature and the third feature to obtain a first fused feature includes: Performing a dilated convolution process with a preset expansion rate on the fourth feature and the third feature respectively to obtain intermediate features corresponding to the features; Performing convolution processing on the fourth feature and the third feature respectively to obtain convolution features corresponding to the features; Based on the fusion of the intermediate features and convolutional features corresponding to the targeted features, an enhanced feature corresponding to the targeted features is obtained; After performing up-pooling processing on the enhanced feature corresponding to the fourth feature, it is concatenated with the enhanced feature corresponding to the third feature to obtain the first fused feature.

10. The method according to claim 1, characterized in that The detecting a target text area in each chip image from the chip image based on the target fusion feature includes: The target fusion features are input into the output layer of the pre-trained text detection network for processing to obtain multiple candidate text regions; A target text region is screened out from the multiple candidate text regions through a non-maximum suppression operation.

11. A chip identifier recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a chip image; the chip image includes chip images of a plurality of chips of different specifications, each of which is an image of the chip surface where the chip identifier is located; The region detection module is used to extract features in sequence through a plurality of feature extraction units connected in sequence based on the chip image, to obtain a first feature and a second feature extracted in sequence by two first feature extraction units connected in sequence among the plurality of feature extraction units in the order of connection, and a third feature and a fourth feature extracted in sequence by two second feature extraction units connected in sequence after the two first feature extraction units in the order of connection; the number of convolution kernels in the convolution layer for convolution processing included in each of the plurality of feature extraction units increases in sequence according to the order of connection of the plurality of feature extraction units; a first fused feature is obtained by fusing the fourth feature with the third feature, and a third fused feature is obtained by fusing the fourth feature with the third feature based on the first fused feature. The first feature and the second feature are fused to obtain a target fused feature; the feature extracted by each first feature extraction unit is obtained by performing a convolution operation on the input of the first feature extraction unit and then performing an inversion operation, and is based on the feature obtained by the convolution operation and the feature obtained by the inversion operation; the feature extracted by each second feature extraction unit is obtained by performing a plurality of different convolution operations on the input of the second feature extraction unit and based on the features extracted by the plurality of convolution operations; the sizes of the features extracted by the plurality of convolution operations are different; based on the target fused feature, a target text area in each chip image is detected from the chip image; The recognition module is used to perform text recognition on the target text area of ​​each chip image to obtain an identifier of the chip corresponding to each chip image.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images