Template obtaining method and device, equipment and storage medium

Abstract

The invention provides a template obtaining method and device, equipment and a storage medium, and the method comprises the steps: obtaining an initial image of a to-be-modeled product, the initial image being an image collected by a camera at a production station for the to-be-modeled product; performing region division on the initial image based on a preset rule to generate a plurality of sub-images; inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image; based on the plurality of mask sub-graphs, obtaining a contour mask graph of the to-be-modeled product; obtaining target information in the contour mask image; and obtaining a template image of the to-be-modeled product based on the target information and the mask image. According to the scheme, modeling is carried out on the physical product, multi-dimensional region division is carried out on the initial image of the physical product, and high-precision and full-automatic generation of the physical product template is finally realized in cooperation with mask graph generation and multi-level screening processing based on the rapid segmentation model, so that the modeling accuracy and industrial application suitability are remarkably improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to a template obtaining method, apparatus, device and storage medium. Background Art

[0002] In the field of image detection, template establishment is crucial. It provides an accurate reference standard for image detection, effectively improving detection efficiency. It also reduces misjudgments and missed detections by precisely depicting target features, thereby enhancing detection accuracy. In machine learning model training, high-quality templates provide a basis for labeling and classification, helping the model learn effective features. Therefore, well-established templates lay the foundation for subsequent image analysis and processing, result evaluation and verification, and other tasks, supporting the efficient operation of the entire image detection process. Summary of the Invention

[0003] The present disclosure provides a template obtaining method, apparatus, device and storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a template obtaining method is provided, the method comprising:

[0005] Obtaining an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station;

[0006] Dividing the initial image into regions based on preset rules to generate multiple sub-images;

[0007] Inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image;

[0008] Obtaining a contour mask image of the product to be modeled based on the plurality of mask sub-images;

[0009] Acquiring target information in the contour mask image;

[0010] A template image of the product to be modeled is obtained based on the target information and the mask image.

[0011] In one embodiment, the sub-images include a first type of sub-image and a second type of sub-image; and the region division of the initial image based on a preset rule to generate the plurality of sub-images includes:

[0012] Divide the initial image into N equal parts along the horizontal direction to generate a first type of sub-image, where N is greater than or equal to 3;

[0013] At the seams of adjacent equally divided areas, areas of preset width are cropped as the second type of sub-images.

[0014] In one possible implementation, the mask sub-graph includes a first type of mask sub-graph and a second type of mask sub-graph;

[0015] Inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image comprises:

[0016] Inputting the first type of sub-image into a segmentation network to obtain an image embedding vector of the first type of sub-image; decoding the image embedding vector of the first type of sub-image to obtain a first type of mask sub-image;

[0017] The second type of sub-image is input into the segmentation network to obtain the image embedding vector of the second type of sub-image; and the image embedding vector of the second type of sub-image is decoded to obtain the second type of mask sub-image.

[0018] In one possible implementation, obtaining the contour mask image of the product to be modeled based on the plurality of mask sub-images includes:

[0019] Processing each of the mask sub-images;

[0020] A contour mask image of the product to be modeled is obtained based on each processed mask sub-image.

[0021] In one embodiment, processing each of the mask sub-images includes:

[0022] Performing connected domain analysis on the mask subgraph to identify independent closed contour regions as candidate regions;

[0023] Retain candidate regions whose area is greater than or equal to the first preset threshold as target candidate regions;

[0024] retaining target candidate regions whose ratio of the intersection area with the valid mask region to the area of ​​the candidate region is less than a second preset threshold, wherein the valid mask region is the region determined to belong to the target component of the product to be modeled;

[0025] Target candidate regions whose distance from the edge of the mask sub-image is greater than or equal to a third preset threshold are retained.

[0026] In one possible implementation, obtaining the contour mask image of the product to be modeled based on the plurality of mask sub-images includes:

[0027] The mask sub-images corresponding to the first type of sub-images are spliced ​​into the basic mask image;

[0028] The target area intersecting with the preset cutting line is filtered from the mask sub-image corresponding to the second type of sub-image, and covered to the corresponding position of the basic mask image to obtain the contour mask image of the product to be modeled; the preset cutting line is the cutting boundary line when generating the second type of sub-image on the initial image.

[0029] In one embodiment, obtaining target information of a target area in the contour mask image includes:

[0030] Acquire a target component area based on the contour mask image, where the target component belongs to the product to be modeled;

[0031] Perform feature detection on the target component area to obtain target information.

[0032] In one embodiment, obtaining the template image of the product to be modeled based on the target information and the mask image includes:

[0033] Binarizing the target information to obtain features of the target component;

[0034] The features of the target component are superimposed on the mask image to obtain a template image of the product to be modeled.

[0035] According to a second aspect of the present disclosure, a template obtaining device is provided, the device comprising:

[0036] A first acquisition module is configured to obtain an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station;

[0037] A segmentation module, configured to divide the initial image into regions based on preset rules to generate a plurality of sub-images;

[0038] a processing module, configured to input the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image;

[0039] A splicing module, configured to obtain a contour mask image of the product to be modeled based on a plurality of the mask sub-images;

[0040] A second acquisition module is used to acquire target information in the contour mask image;

[0041] A template generation module is used to obtain a template image of the product to be modeled based on the target information and the mask image.

[0042] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0043] at least one processor; and

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0046] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0047] The present disclosure provides a template acquisition method, device, equipment and storage medium. The method obtains an initial image of a product to be modeled, where the initial image is an image of the product to be modeled captured by a camera at a production station. The initial image is then divided into regions based on preset rules to generate multiple sub-images. The multiple sub-images are then input into a segmentation network to obtain a mask sub-image corresponding to each sub-image. The contour mask image of the product to be modeled is then obtained based on the multiple mask sub-images. The target information in the contour mask image is then obtained. Finally, the template image of the product to be modeled is obtained based on the target information and the mask image. This solution models physical products. By performing multi-dimensional region division on the initial image of the physical product, combined with mask image generation and multi-level screening processing based on a fast segmentation model, high-precision, fully automated physical product template generation is ultimately achieved. The method is applicable to various product types such as keyboards, electronic product key panels, and mechanical parts, significantly improving modeling accuracy and adaptability to industrial applications.

[0048] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0050] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0051] Figure 1 A schematic diagram showing a flow chart of a template obtaining method according to an embodiment of the present disclosure is shown;

[0052] Figure 2 A schematic diagram showing an initial image of a keyboard according to an embodiment of the present disclosure is shown;

[0053] Figure 3 A schematic diagram of cutting out a keyboard according to an embodiment of the present disclosure is shown;

[0054] Figure 4shows a schematic diagram of the second type of sub-image according to an embodiment of the present disclosure;

[0055] Figure 5 A schematic diagram of a mask sub-image according to an embodiment of the present disclosure is shown;

[0056] Figure 6 A schematic structural diagram of a template obtaining device according to an embodiment of the present disclosure is shown;

[0057] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0059] Template matching plays an important role in the field of industrial automation. For example, on the production line, products often need to be inspected and identified. Template matching can quickly and accurately detect and identify defects or specific features in products. For example, in the automated optical inspection (AOI) of computer keyboards, it is necessary to check whether each key of the keyboard is correctly installed and whether there are any defects, such as missing, skewed, or printed keycaps. The keyboard template defines the "ideal state" of the keyboard (such as key layout, size, character printing, color, etc.). During inspection, defects can be quickly identified by comparing the differences between the actual product and the template. Therefore, establishing a high-quality template image is the core of ensuring detection accuracy, efficiency, and reliability.

[0060] In the current related technologies, the establishment of template images mainly includes two modeling methods: on-site modeling and design drawing-based (Model Based Definition, MBD). However, the on-site modeling method requires manual marking of the detection area (ROI), which is time-consuming; on the other hand, fluctuations in on-site conditions such as lighting and camera angles can lead to modeling errors. Based on MBD modeling, there will be deviations between the design and actual production, and the subsequent low accuracy of template matching will occur due to untimely updates of drawings. Based on this, the present disclosure provides a template acquisition method.

[0061] like Figure 1 A schematic diagram of a process for obtaining a template provided by an embodiment of the present disclosure is shown, and the method includes:

[0062] S1. Obtain an initial image of a product to be modeled, where the initial image is an image of the product to be modeled captured by a camera at a production station.

[0063] Industrial cameras are deployed at production workstations. The camera's shooting angle, focal length, and exposure parameters are set based on the size, shape, and production process of the product to be modeled, ensuring that the captured image is clear and fully covers the product. When the product moves along the production line to a specific workstation, triggering a capture command, or when sensors detect the product's arrival, the camera captures an initial image containing background information about the product to be modeled and its surrounding production environment. Furthermore, you can select a ROI (Region of Interest) and perform a normalization operation to crop the area of ​​the product to be modeled, generating the initial image of the product.

[0064] Taking the laptop keyboard as an example, the product to be modeled is set to a bird's-eye view according to the size of the laptop keyboard and the reflective properties of the keyboard surface. The camera is set to a low-angle shot, the focal length is adjusted to clearly capture the keycap characters and edges, and the exposure parameters are adapted to the contrast between the black keycaps and white characters on the keyboard. The camera shoots and obtains an image containing the entire laptop keyboard and the surrounding background. The letters, symbols on the keycaps and the overall outline of the keyboard are clearly visible in the image. In order to facilitate image processing, it is first necessary to select the ROI area and complete the area's normalization, crop the keyboard area, and obtain the initial image. Figure 2 Shown is a diagram of an initial image of a keyboard.

[0065] S2. Divide the initial image into regions based on preset rules to generate multiple sub-images.

[0066] The preset rules are formulated based on the structural features of the product to be modeled and the subsequent processing requirements. First, the initial image is analyzed through edge detection, corner detection and other algorithms to identify the outline and key structural positions of the product. If the product has regularly arranged parts (such as a keyboard keycap array), the initial image is divided along the direction of the part arrangement (such as the horizontal direction) to obtain multiple sub-images. If the product structure is irregular, areas of different shapes and sizes can also be delineated manually or through intelligent algorithms based on the functional modules or geometric features of the product to generate corresponding sub-images.

[0067] S3. Input the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image.

[0068] A segmentation network based on the Fast Segmentation Model (FastSAM) is used. This network only retains the image encoder structure to reduce computational complexity and improve processing speed. Each sub-image is input into the segmentation network, which extracts and analyzes the input image features to obtain a mask sub-image corresponding to the sub-image.

[0069] S4. Obtaining a contour mask image of the product to be modeled based on the multiple mask sub-images.

[0070] Multiple mask sub-images are spliced ​​together to obtain a contour mask image.

[0071] S5. Acquire target information in the contour mask image.

[0072] Based on the contour mask, a trained object detection neural network is used to detect features of product components within the mask. By learning from a large amount of labeled data, the neural network can identify target information such as characters, graphics, and textures on product components. For each detected target, information such as its position coordinates within the mask, shape and size, and category label is obtained. This information is stored in a data structure for subsequent processing.

[0073] S6. Obtaining a template image of the product to be modeled based on the target information and the mask image.

[0074] The target information acquired in step S5 is fused with the contour mask image to produce a template image of the product to be modeled. First, the location of each target is accurately marked on the contour mask image based on the target coordinates in the target information. Then, information such as the target's shape, size, and category label is overlaid on the mask image. The fused image can also be post-processed, such as adjusting the image resolution and color mode, and removing unnecessary auxiliary lines and annotations, to ensure that the template image meets actual application requirements, resulting in the final template image of the product to be modeled.

[0075] In the above scheme, an initial image of the product to be modeled is obtained, which is an image captured by a camera at a production station. The initial image is then divided into regions based on preset rules to generate multiple sub-images. The multiple sub-images are then input into a segmentation network to obtain a mask sub-image corresponding to each sub-image. A contour mask image of the product to be modeled is then obtained based on the multiple mask sub-images. The target information in the contour mask image is then obtained. Finally, a template image of the product to be modeled is obtained based on the target information and the mask image. This scheme models physical products by performing multi-dimensional region division on the initial image of the physical product, combined with mask image generation and multi-level screening based on a fast segmentation model, ultimately achieving high-precision, fully automated physical product template generation. This scheme is suitable for a variety of product types, such as keyboards, electronic product key panels, and mechanical parts, significantly improving modeling accuracy and adaptability to industrial applications.

[0076] In one example, the sub-images include a first type of sub-image and a second type of sub-image; an initial image is divided into regions based on a preset rule to generate a plurality of sub-images, including:

[0077] Divide the initial image into N equal parts along the horizontal direction to generate a first type of sub-image, where N is greater than or equal to 3;

[0078] At the seams of adjacent equally divided areas, areas of preset width are cropped as the second type of sub-images.

[0079] If the product has regularly arranged components, such as an array of keyboard keycaps, the initial image is divided into N equal parts along the component arrangement direction, such as horizontally, where N ≥ 3, to generate a first type of sub-image covering the main product area. At the seams between adjacent equally divided areas, regions of a preset width are cropped to generate a second type of sub-image, enhancing segmentation accuracy in the seam area. In one example, the original, uncropped image can also be retained as a third type of sub-image to handle special structures or fully display the product.

[0080] Still taking the laptop keyboard as an example, Figure 3 The following is a schematic diagram of keyboard cropping. The initial image is divided into three equal parts along the horizontal direction of the keyboard to obtain three first-type sub-images, covering the keycap bodies in the left, middle, and right areas of the keyboard respectively. At the seams of adjacent equally divided areas, an area with the same width as the first-type sub-image is cropped. Assuming the cropping width is 6cm, the seam is used as the central axis and extended 3cm on both sides of the seam to generate two second-type sub-images for subsequent precise segmentation of the gaps between the keycaps, as shown in the figure below. Figure 4 The following diagram shows a schematic diagram of the second sub-image. The original image is retained as the third sub-image to ensure that special long keycap structures such as the space bar and enter key are fully represented. For example, during the segmentation process, areas such as the F key area and the numeric keypad area are rationally divided into different sub-images, ensuring that each sub-image contains clear keycap information.

[0081] In one example, the mask sub-images include a first type of mask sub-image and a second type of mask sub-image; and inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image includes:

[0082] Inputting the first type of sub-image into a segmentation network to obtain an image embedding vector of the first type of sub-image; decoding the image embedding vector of the first type of sub-image to obtain a first type of mask sub-image;

[0083] The second type of sub-image is input into the segmentation network to obtain the image embedding vector of the second type of sub-image; and the image embedding vector of the second type of sub-image is decoded to obtain the second type of mask sub-image.

[0084] The multiple sub-images generated above are input into the segmentation network, which extracts the underlying texture features (such as the convexity and concavity of the characters) and high-level semantic features (such as the shape and arrangement of the keycaps) of each sub-image. After being mapped by the decoder, each sub-image generates a corresponding mask sub-image. Figure 5The figure shows a schematic diagram of a mask sub-image, where white pixels indicate the position and outline of the keycaps, and black pixels represent the background. For example, in the mask sub-image corresponding to the sub-image containing the letter key area, each letter keycap is accurately outlined, distinguishing it from the black background area.

[0085] In one example, obtaining the contour mask image of the product to be modeled based on the multiple mask sub-images includes:

[0086] Processing each of the mask sub-images;

[0087] A contour mask image of the product to be modeled is obtained based on each processed mask sub-image.

[0088] In one example, processing each of the mask sub-images includes:

[0089] Performing a connected domain analysis on the mask subgraph, identifying each independent closed contour region as a candidate region, and determining the area of ​​each candidate region;

[0090] Retain candidate regions whose area is greater than or equal to the first preset threshold as target candidate regions;

[0091] retaining target candidate regions whose ratio of the intersection area with the valid mask region to the area of ​​the candidate region is less than a second preset threshold, wherein the valid mask region is the region determined to belong to the target component of the product to be modeled;

[0092] Target candidate regions whose distance from the edge of the mask sub-image is greater than or equal to a third preset threshold are retained.

[0093] Still taking the laptop keyboard as an example, a connected domain analysis is performed on each mask sub-graph, and all independent closed contour areas are identified as candidate areas. For example, in a certain mask sub-graph, there may be a single keycap contour area (such as Q, O), a small area formed by misjudgment of the gap between the keycaps, etc. By calculating the pixel area of ​​each candidate area, the candidate areas with an area greater than or equal to the first preset threshold are retained as target candidate areas, and the candidate areas with an area less than the first preset threshold are deleted. The first preset threshold can be determined based on the size of the product to be modeled. For example, in a laptop keyboard, the first preset threshold can be set to 1 / 10 of the keycap area. By eliminating these extremely small areas formed by noise or segmentation errors, interference with subsequent processing is avoided.

[0094] For each target candidate area, calculate the ratio of its intersection area with the valid mask area, where the valid mask area is the area that has been determined to belong to the target component of the product to be modeled, such as the area that has been determined to be the valid keycap. Target candidate areas with a ratio less than the second preset threshold are retained, and target candidate areas with a ratio exceeding the second preset threshold are deleted. Assuming that the second preset threshold is 30%, the target candidate areas with a ratio of intersection area with the valid mask area greater than or equal to 30% are deleted, and the target candidate areas are determined to be duplicate segmentation areas. For example, if the intersection area of ​​a target candidate area with the determined keycap area is too large, it means that the area may be the same keycap divided into multiple parts, and the duplicate parts need to be removed.

[0095] Calculate whether the distance between the target candidate area and the edge of the mask sub-image is less than the third preset threshold. Assuming that the third preset threshold is 20 pixels, if it is less than 20 pixels, the target candidate area is determined to be background interference and deleted, and the target candidate area with a distance greater than or equal to 20 pixels from the edge of the mask sub-image is retained. If the horizontal coordinate on the right side of the candidate area is less than 20 pixels close to the right edge of the mask sub-image, the candidate area is regarded as a space bar. For another example, if most of the edges of a target candidate area are aligned with the boundary of the mask sub-image and the distance is less than 20 pixels, it is likely that the background (such as a touchpad) is mistakenly judged as a keycap and should be removed.

[0096] In one example, obtaining a contour mask image of the product to be modeled based on the plurality of mask sub-images includes:

[0097] The mask sub-images corresponding to the first type of sub-images are spliced ​​into the basic mask image;

[0098] The target area intersecting with the preset cutting line is filtered from the mask sub-image corresponding to the second type of sub-image, and covered to the corresponding position of the basic mask image to obtain the contour mask image of the product to be modeled; the preset cutting line is the cutting boundary line when generating the second type of sub-image on the initial image.

[0099] First, the mask sub-images corresponding to the first type of sub-images are spliced ​​according to their original positional relationship in the initial image to form a basic mask image. The preset cropping line is the cropping boundary line when generating the second type of sub-image on the initial image. Based on the preset cropping line, the target areas that intersect with the preset cropping line are screened out from the mask sub-images corresponding to the second type of sub-images. These target areas usually contain key information of product components at the segmentation seams, such as the complete outline of the component edge. The screened target areas are overlaid on the corresponding positions of the basic mask image, and through pixel-level fusion operations, the possible missing component segmentations at the seams of the basic mask image are filled, thereby obtaining a more complete contour mask image of the product to be modeled.

[0100] In one example, obtaining target information of a target area in the contour mask image includes:

[0101] Acquire a target component area based on the contour mask image, where the target component belongs to the product to be modeled;

[0102] Perform feature detection on the target component area to obtain target information.

[0103] In the generated contour mask of the laptop keyboard, the pixel information of the mask is used to identify independent areas composed of all white pixels, and these areas are determined as keycap target component areas, including letter keys, number keys, function keys, etc. A pre-trained target detection neural network (such as the DetNet model) is used to perform feature detection on each keycap area. During the training process of the target detection neural network, a large amount of laptop keyboard image data with keycap characters, shapes, and positions annotated is used to enable the neural network to learn the characteristic patterns of different keycaps. For example, the keycaps of keyboards of multiple laptop models are cut off and saved, including gray keycaps, ThinkPad keycaps, NEC model keycaps, and white keycaps, and then they are annotated, and icons and characters are marked as a type of feature. In order to increase the robustness of the character and graphic detection algorithm, targeted enhancements can also be made to the data set. Taking into account the differences that may be caused by the different keyboard colors and C-side colors of different models, an inverted color operation is introduced:

[0104] G(i,j)=255-F(i,j) Formula (1)

[0105] In formula (1), (F(i, j) represents the pixel value of the original image at coordinate ((i, j), and G(i, j) represents the pixel value at the same coordinate (i, j) after processing. Formula (1) inverts the brightness values ​​of all pixels in the image by subtracting the original pixel value from 255. For example, the black area with an original pixel value of 0 becomes 255 (white) after calculation; the white area with an original pixel value of 255 becomes 0 (black). In the notebook keyboard modeling scenario, for a keyboard with black keycaps and white characters, the inversion will turn it into white keycaps and black characters, which can enable the algorithm to learn the features of keycaps and characters from different perspectives and reduce the detection bias caused by keyboard color differences.

[0106] Secondly, a random brightness change operation is added to avoid the influence of different lighting on the ability to detect characters on the keycaps:

[0107] G(i,j)=F(i,j)*random(0.5,1.5) Formula (2)

[0108] In formula (2), (F(i,j) represents the pixel value of the original image at the coordinate ((i,j), and G(i,j) represents the pixel value at the same coordinate (i,j) after processing. random(0.5,1.5) represents randomly generating a value between 0.5 and 1.5. This formula multiplies the pixel value of each pixel in the original image by a random number between 0.5 and 1.5. When the random number is less than 1, the pixel value becomes darker and the overall brightness of the image decreases; when the random number is greater than 1, the pixel value becomes brighter and the overall brightness of the image increases. By simulating keyboard images under different lighting conditions in this way, the neural network used for keycap character detection can adapt to various lighting intensities, enhance the robustness of the model in different lighting environments, and avoid the problem of difficult character recognition due to different lighting conditions.

[0109] In order to avoid the image out of focus caused by the on-site focus problem in the later stage, which affects the modeling ability, the image sharpness change based on the Laplace operator is added:

[0110] G(i,j)=F(i,j)+λ×LPF(F(i,j))-LPF(F(i,j)) Formula (3)

[0111] In formula (3), λ is an adjustable weight parameter used to control the degree of sharpening; LPF represents the Laplacian operator, which is a second-order derivative operator used to detect edges and details in images. In this formula, LPF(F(i,j)) calculates the Laplace transform result of the original image, and LPF(F(i,j)-LPF(F(i,j)) obtains the high-frequency detail information of the image. By multiplying this high-frequency information by the weight and adding it to the original image (F(i,j), the edges and details of the image can be enhanced, making the blurred image clearer. In the modeling of laptop keyboards, the image out-of-focus problem caused by on-site focusing problems can be solved, making the keycap edges and characters clearer, and improving the accuracy of modeling.

[0112] In addition, conventional processing histogram equalization operations and overexposure simulation based on fixed thresholds are also added.

[0113] In one example, obtaining the template image of the product to be modeled based on the target information and the mask image includes:

[0114] Binarizing the target information to obtain features of the target component;

[0115] The features of the target component are superimposed on the mask image to obtain a template image of the product to be modeled.

[0116] Convert the characters on each keycap into a binary bitmap of a standard font. For example, for the character "A", generate a 32×32 pixel binary image with the character area being white (255) and the background being black (0). Special symbols, such as ">", are converted from vector graphics into binary images to ensure clear edges of the symbols. Convert the position coordinates (x, y) and size (w, h) of the keycap into normalized coordinates relative to the mask image. According to the position coordinates of the keycap in the mask image, accurately place the binary character bitmap at the corresponding position, and finally obtain a template image. This template image can be directly used in industrial application scenarios such as keyboard production quality inspection and keycap printing design.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides a template obtaining device, such as Figure 6 The figure shows a schematic structural diagram of the device, which includes:

[0118] A first acquisition module 10 is configured to obtain an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station;

[0119] A segmentation module 20 is configured to divide the initial image into regions based on a preset rule to generate a plurality of sub-images;

[0120] A processing module 30 is configured to input the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image;

[0121] A splicing module 40 is configured to obtain a contour mask image of the product to be modeled based on the plurality of mask sub-images;

[0122] A second acquisition module 50 is used to acquire target information in the contour mask image;

[0123] The template generation module 60 is configured to obtain a template image of the product to be modeled based on the target information and the mask image.

[0124] In one example, the segmentation module 20 is further configured to:

[0125] Divide the initial image into N equal parts along the horizontal direction to generate a first type of sub-image, where N is greater than or equal to 3;

[0126] At the seams of adjacent equally divided areas, areas of preset width are cropped as the second type of sub-images.

[0127] In one example, the processing module 30 is further configured to:

[0128] Inputting the first type of sub-image into a segmentation network to obtain an image embedding vector of the first type of sub-image; decoding the image embedding vector of the first type of sub-image to obtain a first type of mask sub-image;

[0129] The second type of sub-image is input into the segmentation network to obtain the image embedding vector of the second type of sub-image; and the image embedding vector of the second type of sub-image is decoded to obtain the second type of mask sub-image.

[0130] In one example, the processing module 30 is further configured to:

[0131] Processing each of the mask sub-images includes:

[0132] Performing a connected domain analysis on the mask subgraph, identifying each independent closed contour region as a candidate region, and determining the area of ​​each candidate region;

[0133] Deleting candidate regions whose areas are smaller than a first preset threshold;

[0134] If the ratio of the intersection area of ​​the current candidate region and the target region to the area of ​​the current candidate region exceeds a second preset threshold, the candidate region is determined to be repeatedly segmented and deleted;

[0135] If the distance between the candidate region and the edge of the mask sub-image is less than a third preset threshold, the candidate region is determined to be background interference and is deleted.

[0136] In one example, the splicing module 40 is further configured to:

[0137] The mask sub-images corresponding to the first type of sub-images are spliced ​​into the basic mask image;

[0138] The target area intersecting with the preset cutting line is filtered from the mask sub-image corresponding to the second type of sub-image, and covered to the corresponding position of the basic mask image to obtain the contour mask image of the product to be modeled; the preset cutting line is the cutting boundary line when generating the second type of sub-image on the initial image.

[0139] In one example, the second obtaining module 50 is further configured to:

[0140] Acquire a target component area based on the contour mask image, where the target component belongs to the product to be modeled;

[0141] Perform feature detection on the target component area to obtain target information.

[0142] In one example, the template generation module 60 is further configured to:

[0143] Binarizing the target information to obtain features of the target component;

[0144] The features of the target component are superimposed on the mask image to obtain a template image of the product to be modeled.

[0145] According to an embodiment of the present disclosure, there is further provided an electronic device, including:

[0146] at least one processor; and

[0147] a memory communicatively connected to the at least one processor; wherein,

[0148] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0149] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0150] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0151] Figure 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0152] like Figure 7 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0153] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0154] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the template acquisition method. For example, in some embodiments, the template acquisition method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the template acquisition method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the template acquisition method by any other appropriate means (e.g., by means of firmware).

[0155] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0159] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0160] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0161] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0162] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0163] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A template obtaining method, characterized in that: The method comprises: Obtaining an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station; Dividing the initial image into regions based on preset rules to generate multiple sub-images; Inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image; Obtaining a contour mask image of the product to be modeled based on the plurality of mask sub-images; Acquiring target information in the contour mask image; A template image of the product to be modeled is obtained based on the target information and the mask image.

2. The template obtaining method according to claim 1, characterized in that: The sub-images include a first type of sub-image and a second type of sub-image; The step of dividing the initial image into regions based on a preset rule to generate a plurality of sub-images includes: Divide the initial image into N equal parts along the horizontal direction to generate a first type of sub-image, where N is greater than or equal to 3; At the seams of adjacent equally divided areas, areas of preset width are cropped as the second type of sub-images.

3. The template obtaining method according to claim 2, characterized in that: The mask sub-images include a first type of mask sub-image and a second type of mask sub-image; Inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image comprises: Inputting the first type of sub-image into a segmentation network to obtain an image embedding vector of the first type of sub-image; decoding the image embedding vector of the first type of sub-image to obtain a first type of mask sub-image; The second type of sub-image is input into the segmentation network to obtain the image embedding vector of the second type of sub-image; and the image embedding vector of the second type of sub-image is decoded to obtain the second type of mask sub-image.

4. The template obtaining method according to claim 1, characterized in that: The step of obtaining the contour mask image of the product to be modeled based on the plurality of mask sub-images includes: Processing each of the mask sub-images; A contour mask image of the product to be modeled is obtained based on each processed mask sub-image.

5. The template obtaining method according to claim 4, characterized in that: The processing of each of the mask sub-images includes: Performing connected domain analysis on the mask subgraph to identify independent closed contour regions as candidate regions; Retain candidate regions whose area is greater than or equal to the first preset threshold as target candidate regions; retaining target candidate regions whose ratio of the intersection area with the valid mask region to the area of ​​the candidate region is less than a second preset threshold, wherein the valid mask region is the region determined to belong to the target component of the product to be modeled; Target candidate regions whose distance from the edge of the mask sub-image is greater than or equal to a third preset threshold are retained.

6. The template obtaining method according to claim 2, characterized in that: The step of obtaining the contour mask image of the product to be modeled based on the plurality of mask sub-images includes: The mask sub-images corresponding to the first type of sub-images are spliced ​​into the basic mask image; The target area intersecting with the preset cutting line is filtered from the mask sub-image corresponding to the second type of sub-image, and covered to the corresponding position of the basic mask image to obtain the contour mask image of the product to be modeled; the preset cutting line is the cutting boundary line when generating the second type of sub-image on the initial image.

7. The template obtaining method according to claim 1, characterized in that: The acquiring target information of the target area in the contour mask image includes: Acquire a target component area based on the contour mask image, where the target component belongs to the product to be modeled; Perform feature detection on the target component area to obtain target information.

8. The template obtaining method according to claim 1, characterized in that: The obtaining of the template image of the product to be modeled based on the target information and the mask image includes: Binarizing the target information to obtain features of the target component; The features of the target component are superimposed on the mask image to obtain a template image of the product to be modeled.

9. A template obtaining device, characterized in that: The device comprises: A first acquisition module is configured to obtain an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station; A segmentation module, configured to divide the initial image into regions based on preset rules to generate a plurality of sub-images; a processing module, configured to input the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image; A splicing module, configured to obtain a contour mask image of the product to be modeled based on a plurality of the mask sub-images; A second acquisition module is used to acquire target information in the contour mask image; A template generation module is used to obtain a template image of the product to be modeled based on the target information and the mask image.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform: Obtaining an initial image of the product to be modeled, wherein the initial image is an image of the product to be modeled captured by a camera at a production station; Dividing the initial image into regions based on preset rules to generate multiple sub-images; Inputting the plurality of sub-images into a segmentation network to obtain a mask sub-image corresponding to each sub-image; Obtaining a contour mask image of the product to be modeled based on the plurality of mask sub-images; Acquiring target information in the contour mask image; A template image of the product to be modeled is obtained based on the target information and the mask image.

Images