Object positioning method, device and storage medium

Abstract

An object positioning method, device and storage medium, the method comprising: establishing a detection model based on reference data as a training set; the reference data is a plurality of sample images of a sealed box; wherein the capacity of the sealed box for accommodating the target object is M, and the number of sample images is less than 2 M ; Use a camera to obtain a detection image of the sealed box to be inspected; divide the detection image evenly into a specified number of detection areas according to a predetermined method; extract the image of the corresponding position of each detection area in the detection image as a local image; input each local image into the detection model in turn; obtain the quantity information and position information of the target objects in the sealed box based on the statistics of multiple detection results output by the detection model.

Description

Technical Field

[0001] The invention relates to an object positioning method, device and storage medium for an object contained in a sealed box. Background Art

[0002] Certain objects, such as wafers and semiconductor structures, are often stored in confined spaces during transportation to prevent contamination from the surrounding environment. Existing inspection methods typically rely on manual visual inspection to determine the number and position of objects within a confined space from different angles, ensuring that the objects are not contaminated. However, the dense arrangement of objects within a confined space, coupled with the varying shapes of the confined space, can easily lead to omissions or misjudgments during manual inspection. Summary of the Invention

[0003] The main purpose of the present invention is to provide an object positioning method, device and storage medium, aiming to solve the problem of easy omission or misjudgment by manual visual inspection in the prior art.

[0004] An object positioning method is used to detect the number and position of target objects in a sealed box, the object positioning method comprising:

[0005] The detection model is established based on the benchmark data as a training set; the benchmark data is a plurality of sample images; wherein the capacity of the sealed box is M, and the number of the sample images is less than 2 M ;

[0006] Using a camera to obtain a detection image of the sealed box to be detected;

[0007] Evenly dividing the detection image into a specified number of detection areas according to a predetermined method;

[0008] Extracting an image at a position corresponding to each detection area in the detection image as a local image;

[0009] Inputting each of the local images into the detection model in sequence;

[0010] The quantity information and position information of the target objects in the sealed box are obtained based on the multiple detection results output by the detection model.

[0011] Preferably, the specified number is N, and N is less than M; the number of sample images is 2 （M / N） .

[0012] Preferably, the step of establishing a detection model based on the benchmark data as a training set includes:

[0013] Evenly dividing the image acquisition area according to the predetermined method to form the specified number of first detection areas;

[0014] extracting one of the first detection areas as a first reference area and acquiring a plurality of images corresponding to different numbers of target objects at different positions in the first reference area as the sample images;

[0015] extracting a second detection area in the image area as a second reference area and acquiring a plurality of images of different numbers of target objects in different positions in the second reference area as supplementary images;

[0016] The detection model is established based on the sample image and the supplementary image as the training set.

[0017] Preferably, the second detection area is an area that overlaps with two adjacent first detection areas; and the number of the supplementary images is less than or equal to the number of the sample images.

[0018] Preferably, the second detection area is set to be the same size as the first detection area.

[0019] Preferably, the detection model is a deep learning network model.

[0020] Preferably, the sealed box is placed under a designated light source; the designated light source is used to enhance edge features of the target object.

[0021] Preferably, the camera acquires the image of the sealed box from a specified angle; the specified angle is related to the transparency and shape of the sealed box.

[0022] Furthermore, to achieve the above-mentioned objectives, the present invention further provides an object positioning device for detecting the number and position of target objects in a sealed box; the object positioning device includes a processor and a memory, and the processor is configured to implement the following steps when executing a computer program stored in the memory:

[0023] The detection model is established based on the benchmark data as a training set; the benchmark data is a plurality of sample images of a sealed box; wherein the capacity of the sealed box is M, and the number of the sample images is less than 2 M ;

[0024] Using a camera to obtain a detection image of the sealed box to be detected;

[0025] Evenly dividing the detection image into a specified number of detection areas according to a predetermined method;

[0026] Extracting an image at a position corresponding to each detection area in the detection image as a local image;

[0027] Inputting each of the local images into the detection model in sequence;

[0028] The quantity information and position information of the target objects in the sealed box are obtained based on the multiple detection results output by the detection model.

[0029] In addition, to achieve the above-mentioned object, the present invention further provides a storage medium, which is a computer-readable storage medium and stores at least one instruction. When the at least one instruction is executed by a processor, the following steps are implemented:

[0030] The detection model is established based on the benchmark data as a training set; the benchmark data is a plurality of sample images of a sealed box; wherein the capacity of the sealed box is M, and the number of the sample images is less than 2 M ;

[0031] Using a camera to obtain a detection image of the sealed box to be detected;

[0032] Evenly dividing the detection image into a specified number of detection areas according to a predetermined method;

[0033] Extracting an image at a position corresponding to each detection area in the detection image as a local image;

[0034] Inputting each of the local images into the detection model in sequence;

[0035] The quantity information and position information of the target objects in the sealed box are obtained based on the multiple detection results output by the detection model.

[0036] The above-mentioned object positioning method, device and storage medium can reduce the number of sample images in the training set corresponding to the detection model. At the same time, by dividing the detection image of the sealed box to be detected into multiple detection areas, the local image corresponding to each detection area is extracted and input into the detection model. By adopting the above-mentioned segmentation method, the processing speed of the detection model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 This is a functional module diagram of the object positioning device of the present invention.

[0039] Figure 2 for Figure 1 Functional module diagram of the object positioning system described in .

[0040] Figure 3 for Figure 1 Schematic diagram of the positional relationship between the camera, the sealed box and the designated light source.

[0041] Figure 4 Schematic diagram of the detection image.

[0042] Figure 5 The figure is a flowchart of an object positioning method.

[0043] Figure 6 for Figure 5 Detailed flow chart of step S11 in FIG.

[0044] Description of main component symbols

[0045]

[0046] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0048] The terms "first," "second," and "third," etc., in the present description and accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "comprise," "comprising," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to the process, method, product, or apparatus.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0050] The specific implementation of the object positioning method of the present invention is described below with reference to the accompanying drawings.

[0051] Please refer to Figure 1 as well as Figure 2The present invention provides a module schematic diagram of an object positioning device 100 and a module schematic diagram of the object positioning system. The object positioning device 100 is used to photograph a target object 201 in a sealed box 200, input the obtained image to be inspected into a detection model, and determine the number and position of the target objects 201 in the sealed box based on the output structure of the detection model. In at least one embodiment of the present invention, the sealed box 200 is a wafer box or a wafer transfer box. The sealed box 200 can be made of a transparent material or a translucent material. The target object 201 is a wafer.

[0052] The object positioning device 100 includes a memory 102 , a processor 103 , a communication bus 104 , and a camera 106 .

[0053] The memory 102 is used to store program code. The memory 102 can be a circuit with storage functionality within an integrated circuit that does not have a physical form, such as a memory stick, a TF card (Trans-flash card), a smart media card, a secure digital card, a flash memory card, or other storage device. The memory 102 can communicate data with the processor 103 via the communication bus 104. The memory 102 can include the object positioning system 1 and the operating system 2.

[0054] The operating system 2 is a program for managing and controlling hardware and software resources, and supports the operation of the object positioning system 1 and other software and / or programs.

[0055] The processor 103 may include one or more microprocessors or digital processors. The processor 103 may call the program code stored in the memory 102 to execute related functions. For example, Figure 2 Each module described is a program code stored in the memory 102 and executed by the processor 103 to implement an object positioning method. The processor 103, also known as the central processing unit (CPU), is a large-scale integrated circuit that serves as both the computing core and the control unit.

[0056] The communication bus 104 performs data communication with the memory 102 and the processor 103 .

[0057] The camera 106 is used to photograph the target object 201 contained in the sealed box 200 to form a sample image and a detection image.

[0058] Please also refer to Figure 2 , which is a module diagram of the object positioning system 1. The object positioning system 1 includes:

[0059] The model building module 10 is used to build a detection model based on the benchmark data as a training set.

[0060] In at least one embodiment of the present invention, the reference data is a plurality of sample images of the target object 201 of different numbers and positions contained in the sealed box 200. The reference data includes a plurality of sample images and a plurality of supplementary images. The number of the supplementary images is less than or equal to the number of the sample images. The capacity of the sealed box 200 is M, the specified number is N, and N is less than M; the number of the sample images is 2 （M / N） Taking the capacity of the sealed box as 25 as an example, the number of sample images is 2 5 =32; the number of supplementary images can be 3. Both the sample image and the supplementary perspective image can identify the target object 201 and the position of the target object 201 within the sealed box 200. The specified number can be set based on user needs. The detection model is a deep learning network model.

[0061] The model building module 10 further divides the image acquisition area into the specified number of first detection areas A (such as Figure 4 As shown in FIG1 ), extract a first detection area A as a first reference area and obtain multiple images corresponding to different numbers of target objects 201 at different positions in the first reference area as the sample images, extract a second detection area B (as shown in FIG1 ) in the image acquisition area. Figure 4 As shown in FIG. 1 , a plurality of images corresponding to different numbers of target objects 201 at different positions within the second reference area are obtained as supplementary images. The detection model is established using the sample images and the supplementary images as the training set. The supplementary images are used to further identify whether the target objects 201 exist in two adjacent first detection areas A. The image acquisition area is an area in the image captured by the camera 106 that is the same size as the sealed box 200.

[0062] In at least one embodiment of the present invention, the second detection area B is overlapped with two adjacent first detection areas A. The second detection area B and the first detection area A are of equal size.

[0063] The image acquisition module 20 is configured to acquire an inspection image of the sealed box 200 to be inspected through the camera 106 .

[0064] In at least one embodiment of the present invention, the sealed box 200 is placed under a designated light source 300 (e.g. Figure 3 ). The designated light source 300 is used to enhance the edge features of the target object 201 within the sealed box 200. The designated light source 300 can be an ultraviolet lamp or a fluorescent lamp, but is not limited thereto. The relative positions of the camera 106, the designated light source 300, and the sealed box 200 are related to the transparency and shape of the sealed box 200. For example, when the upper half of the sealed box 200 is transparent and the target object 201 is positioned from the side, the camera 106 can be located directly above the sealed box 200. The designated light source 300 is located in the upper left corner of the sealed box.

[0065] The image processing module 30 is configured to evenly divide the detection image into the designated number of detection areas according to a predetermined method.

[0066] The extraction module 40 is used to extract the image of each position corresponding to the first detection area A in the detection image as a local image, and input each of the local images into the detection model in turn.

[0067] The statistical module 50 is configured to obtain the quantity information and position information of the target objects in the sealed box based on the multiple detection results output by the detection model.

[0068] In at least one embodiment of the present invention, the quantity information is the total number of the target objects contained in the sealed box and the total number of the target objects corresponding to each partial image. The position information includes the number of the detection area and the position number of the target object within the corresponding detection area.

[0069] The object locating device 100 described above can reduce the number of sample images in the training set corresponding to the detection model. Furthermore, by dividing the image of the sealed box 200 to be inspected, the processing speed of the detection model can be increased. Furthermore, by placing the sealed box 200 under the designated light source 300, the edge features of the target object 201 within the sealed box 201 can be enhanced, thereby improving the accuracy of the detection model's detection results.

[0070] See also Figure 5, which is a flow chart of an object positioning method of the present invention. The object positioning method is applied to the object positioning system 1. The object positioning method photographs the target object 201 in the sealed box 200, inputs the obtained image to be detected into the detection model, and determines the number and position of the target objects 201 in the sealed box based on the output structure of the detection model. In at least one embodiment of the present invention, the sealed box 200 is a wafer box or a wafer transfer box. The sealed box 200 can be made of a transparent material or a translucent material. The target object 201 is a wafer.

[0071] The object positioning method comprises the following steps:

[0072] S11. Establish a detection model based on the benchmark data as a training set.

[0073] In at least one embodiment of the present invention, the reference data is a plurality of sample images of the target object 201 of different numbers and positions contained in the sealed box 200. The reference data includes a plurality of sample images and a plurality of supplementary images. The number of the supplementary images is less than or equal to the number of the sample images. The capacity of the sealed box 200 is M, the specified number is N, and N is less than M; the number of the sample images is 2 （M / N） Taking the capacity of the sealed box as 25 as an example, the number of sample images is 2 5 =32; the number of supplementary images can be 3. Both the sample image and the supplementary perspective image can identify the target object 201 and the position of the target object 201 within the sealed box 200. The specified number can be set based on user needs. The detection model is a deep learning network model.

[0074] Please also refer to Figure 6 , which is a detailed flow chart of step S11. In at least one embodiment of the present invention, the step of establishing a detection model based on the benchmark data as a training set further includes:

[0075] S111, evenly dividing the image acquisition area according to the predetermined method to form the specified number of first detection areas A;

[0076] S112, extracting a first detection area A as a first reference area and obtaining a plurality of images corresponding to different numbers of target objects 201 at different positions in the first area as the sample images;

[0077] S113, extracting a second detection area B in the image acquisition area as a second reference area and acquiring a plurality of images corresponding to different numbers of target objects 201 at different positions in the second reference area as supplementary images;

[0078] S114: Use the sample image and the supplementary image as the detection model of the training set.

[0079] In at least one embodiment of the present invention, the supplementary image is used to further identify whether the target object 201 exists in two adjacent first detection areas A. The image acquisition area is an area in the image acquired by the camera 106 that is as large as the sealed box 200 .

[0080] The second detection area B is an area that overlaps with two adjacent first detection areas A. In at least one embodiment of the present invention, the second detection area B and the first detection area A are of the same size.

[0081] S12 , obtaining a detection image of the sealed box 200 to be detected through the camera 106 .

[0082] In at least one embodiment of the present invention, the sealed box 200 is placed under a designated light source 300 (e.g. Figure 3 ). The designated light source 300 is used to enhance the edge features of the target object 201 within the sealed box 200. The designated light source 300 can be an ultraviolet lamp or a fluorescent lamp, but is not limited thereto. The relative positions of the camera 106, the designated light source 300, and the sealed box 200 are related to the transparency and shape of the sealed box 200. For example, when the upper half of the sealed box 200 is transparent and the target object 201 is positioned from the side, the camera 106 can be located directly above the sealed box 200. The designated light source 300 is located in the upper left corner of the sealed box.

[0083] S13. Evenly divide the detection image into the specified number of detection areas according to a predetermined method.

[0084] S14. Extract an image at a position corresponding to each first detection area A in the detection image as a local image.

[0085] S15. Input each of the local images into the detection model in sequence.

[0086] S16. Obtaining the quantity information and position information of the target objects in the sealed box according to the multiple detection results output by the detection model.

[0087] In at least one embodiment of the present invention, the quantity information is the total number of the target objects contained in the sealed box and the total number of the target objects corresponding to each partial image. The position information includes the number of the detection area and the position number of the target object within the corresponding detection area.

[0088] The object localization method described above can reduce the number of sample images in the training set corresponding to the detection model. Furthermore, by dividing the image of the sealed box 200 to be inspected, the processing speed of the detection model can be increased. Furthermore, by placing the sealed box 200 under the designated light source 300, the edge features of the target object 201 within the sealed box 201 can be enhanced, thereby improving the accuracy of the detection model's detection results.

[0089] The present invention also provides a storage medium. The storage medium is a computer-readable storage medium. The computer-readable storage medium stores computer instructions. The computer instructions can be stored in the memory 102 and, when executed by one or more processors 103, implement the object positioning method described in the above method embodiment, for example Figure 5 S11-S16 shown are not described in detail here.

[0090] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical or other forms.

[0092] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0093] In addition, the functional modules in the various embodiments of the present invention may be integrated into a single processor, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0094] If the integrated modules are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0095] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0096] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An object positioning method for detecting the number and position of target objects in a sealed box, characterized in that: The object positioning method comprises: The detection model is established based on the benchmark data as a training set; the benchmark data is a plurality of sample images of the sealed box; wherein the capacity of the sealed box is M, and the number of the sample images is less than 2 M ; Using a camera to obtain a detection image of the sealed box to be detected; Evenly dividing the detection image into a specified number of detection areas according to a predetermined method; Extracting an image at a position corresponding to each detection area in the detection image as a local image; Inputting each of the local images into the detection model in sequence; Obtaining quantity information and position information of the target objects in the sealed box according to a plurality of detection results output by the detection model; The step of establishing a detection model based on the benchmark data as a training set includes: Evenly dividing the image acquisition area according to the predetermined method to form the specified number of first detection areas; extracting one of the first detection areas as a first reference area and acquiring a plurality of images corresponding to different numbers of target objects at different positions in the first reference area as the sample images; Extracting a second detection area from the image acquisition area as a second reference area and acquiring multiple images of different numbers of target objects at different positions in the second reference area as supplementary images; the second detection area is an area overlapping with two adjacent first detection areas; the number of the supplementary images is less than or equal to the number of the sample images; The detection model is established based on the sample image and the supplementary image as the training set.

2. The object positioning method according to claim 1, wherein: The specified number is N, and N is less than M; the number of sample images is 2 （M / N） .

3. The object positioning method according to claim 1, wherein: The second detection area is set to be the same size as the first detection area.

4. The object positioning method according to claim 1, wherein the detection model is a deep learning network model.

5. The object positioning method according to any one of claims 1 to 3, characterized in that: The sealed box is placed under a designated light source; the designated light source is used to enhance the edge features of the target object.

6. The object positioning method according to any one of claims 1 to 3, characterized in that: The camera acquires an image of the sealed box from a specified angle; the specified angle is related to the transparency and shape of the sealed box.

7. An object positioning device comprising a processor and a memory; characterized in that: The processor is configured to implement the object positioning method according to any one of claims 1 to 6 when executing the computer program stored in the memory.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium storing at least one instruction, and when the at least one instruction is executed by a processor, the object positioning method according to any one of claims 1 to 6 is implemented.

Images