Component positioning method, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of image processing, and particularly relate to a component positioning method, an electronic device and a storage medium. Embodiments of the present application cut a target image into multiple candidate region images for positioning, integrate the initial positioning results in combination with the position information of the candidate region images in the target image, group and merge the candidate bounding rectangles based on the position relationship between the horizontal overlapping region, the vertical overlapping region and the candidate bounding rectangle, effectively associate and fuse the repeated detection results across regions, reduce or even avoid the problems of repeated positioning, missed positioning and boundary truncation caused by image blocking processing, significantly improve the integrity, accuracy and consistency of the component positioning results in the large-size circuit board image, and improve the reliability of component positioning while taking into account the efficiency of large-size image processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a component positioning method, electronic device, and storage medium. Background Technology

[0002] In the field of visual inspection in the electronics industry, the size of PCB boards varies significantly depending on the application scenario, ranging from centimeter-level wearable smart device control modules to meter-level large server motherboards. Since the physical dimensions of large PCB boards typically exceed the field of view (FOV) of conventional industrial cameras, the mainstream industry solution for component registration employs non-overlapping sliding window image acquisition. This method uses a camera to seamlessly capture the entire PCB board, obtaining complete inspection information in the shortest possible time. Subsequently, image stitching algorithms are used to fuse multiple sub-field-of-view images into a complete whole-board image. However, directly inputting ultra-high-resolution whole-board images into the inspection model for inference can easily lead to problems such as excessively long inference times and insufficient graphics card memory, affecting not only the inspection cycle but also potentially causing the inspection task to terminate abnormally.

[0003] To this end, the entire board image is divided into several regions of interest (RoIs) with overlapping areas. Model inference is performed on each RoI separately to obtain component localization results. Finally, the duplicate detection results in the overlapping areas are fused, the local detection results of each RoI are mapped and integrated into the global coordinate system, and finally a unified component localization result for the entire board is obtained, completing the component registration process for the target under test.

[0004] In related technologies, the overlapping region fusion scheme of whole-board image localization methods has defects. Most related technologies adopt a sequential suppression mechanism: if a certain element appears in the overlapping region of a previous region of interest (RoI), then the detection results that overlap with the coordinates of that element in the overlapping region of a subsequent RoI will be suppressed. This mechanism works normally when the element is not adjacent to the boundary of the RoI.

[0005] However, when a small portion of a component falls into the previous region of interest (RoI), while the main component is distributed in the next region of interest (RoI), after sequential suppression fusion, only a small detection box in the previous RoI will be retained, resulting in incomplete component localization and ultimately causing component registration failure. Summary of the Invention

[0006] One objective of this application is to provide a component positioning method, electronic device, and storage medium to improve the solution to the problem of incomplete component positioning in the related art.

[0007] In a first aspect, embodiments of this application provide a component localization method, comprising: acquiring an input target image and image processing parameters, wherein the target image is a complete image of a circuit board to be detected, the target image includes components on the circuit board, and the image processing parameters include a reference outer length, the width and height of the target image, and the width and height of a reference region image; cropping a region image from the target image according to the image processing parameters to obtain multiple candidate region images and the position information of each candidate region image in the target image; inputting the candidate region images into a preset localization model for inference localization to obtain an initial localization result of the component in the candidate region image, the initial localization result including the initial coordinates of the upper left corner vertex and the lower right corner vertex of the initial bounding rectangle of the component; integrating the candidate bounding rectangles into the target image according to the width and height of the candidate region images and the position information of the candidate region images in the target image, the target image including the candidate bounding rectangles. The candidate bounding rectangle is the initial bounding rectangle of the element in the candidate region image. Based on the image processing parameters and the candidate region image, a first bounding rectangle and a second bounding rectangle are determined in the target image. The first bounding rectangle is the bounding rectangle of the horizontally overlapping region, which is the region where adjacent candidate region images overlap in the first axis direction of the target image. The second bounding rectangle is the bounding rectangle of the vertically overlapping region, which is the region where adjacent candidate region images overlap in the second axis direction of the target image. Based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, the candidate bounding rectangles are grouped to obtain multiple grouping results. Each grouping result includes at least one candidate bounding rectangle. The candidate bounding rectangles in the grouping results are merged to obtain the final positioning result of the element. The final positioning result includes the final coordinates of the top left corner vertex and the bottom right corner vertex of the element's final bounding rectangle.

[0008] Secondly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the processor is communicatively connected to the memory, and the memory stores program instructions executable by the processor. When the program instructions are executed by the processor, the electronic device performs the component positioning method provided in the first aspect.

[0009] Thirdly, embodiments of this application provide a computer-readable storage medium storing processor-executable program instructions, which, when executed by a processor, cause the processor to perform the element positioning method provided in the first aspect.

[0010] The embodiments of this application have the following beneficial effects: Unlike related technologies, the embodiments of this application locate the target image by cropping it into multiple candidate region images, and integrate the initial positioning results by combining the position information of the candidate region images in the target image. At the same time, the candidate bounding rectangles are grouped and merged based on the positional relationship between the horizontal overlapping region, the vertical overlapping region and the candidate bounding rectangle, so as to achieve effective association and fusion of cross-regional repeated detection results, reduce or even avoid the problems of repeated positioning, missed positioning and boundary truncation caused by image block processing, significantly improve the integrity, accuracy and consistency of component positioning results in large-size circuit board images, and improve the reliability of component positioning while taking into account the efficiency of large-size image processing. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the related technologies or embodiments will be briefly introduced below. Obviously, the drawings described below only show some embodiments of this application and should not be considered as limiting the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating the application scenarios of the positioning element in some embodiments of this application; Figure 2 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application; Figure 3 This is a flowchart illustrating the component positioning method provided in some embodiments of this application; Figure 4 This is a schematic diagram of cropping candidate region images in some embodiments of this application. Figure 1 ; Figure 5 This is a schematic diagram of the target image provided in some embodiments of this application; Figure 6 This is a schematic diagram of cropping candidate region images in some embodiments of this application. Figure 2 ; Figures 7A to 7C This is a schematic diagram of candidate region images obtained by cropping the target image in some embodiments of this application; Figure 8A This is a schematic diagram of integrating the candidate bounding rectangles in the candidate region image to the target image in some embodiments of this application; Figure 8B This is a schematic diagram showing the determination of the first and second bounding rectangles in the target image in some embodiments of this application; Figure 9This is a schematic diagram of the grouping results obtained by grouping candidate bounding rectangles of the target image in some embodiments of this application; Figure 10 This is a schematic diagram of the merged candidate bounding rectangles in the grouping results of the target image in some embodiments of this application. Detailed Implementation

[0013] To make the objectives and advantages of the embodiments of this application more readily understood, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The detailed description of the embodiments of this application in the accompanying drawings is not intended to limit the scope of protection claimed by this application, but only represents selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be noted that, unless there is a conflict, the various technical features involved in the embodiments of this application described below can be combined with each other, and all are within the protection scope of this application. Furthermore, although functional modules are divided in the device or structural schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," "third," and other similar expressions used herein do not limit the data or execution order, but are only for illustrative purposes and to distinguish identical or similar items with substantially the same function and effect, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features.

[0015] Unless otherwise defined, the technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. It should be understood that the term "and / or" as used in this specification includes any and all combinations of one or more of the listed items.

[0016] This application provides a component localization method. The method involves cropping a target image into multiple candidate region images for localization, integrating the initial localization results with the positional information of the candidate region images within the target image, and grouping and merging the candidate bounding rectangles based on the positional relationship between the horizontal and vertical overlapping regions and the candidate bounding rectangles. This achieves effective association and fusion of cross-regional duplicate detection results, reducing or even avoiding problems such as duplicate localization, missed localization, and boundary truncation caused by image block processing. It significantly improves the integrity, accuracy, and consistency of component localization results in large-size circuit board images, while maintaining the efficiency of large-size image processing and enhancing the reliability of component localization.

[0017] Please see Figure 1 , Figure 1 The diagram illustrates application scenarios of the positioning element in some embodiments of this application.

[0018] like Figure 1 As shown, this application scenario includes an electronic device 100 and a host computer 200, with the electronic device 100 communicating with the host computer 200 via a network. Examples of networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0019] Electronic device 100 includes any suitable type of device such as mobile device, terminal device, communication device, computing device, vehicle-mounted device, user equipment (UE), handheld device, cloud device, and server. For example, electronic device 100 can be a desktop computer, laptop computer, tablet computer, all-in-one computer, server, microcontroller, and single-chip microcomputer.

[0020] The servers include rack servers, blade servers, tower servers, cabinet-type servers, and microservers. The electronic device 100 is equipped with a positioning model 101, which is configured to locate components on a circuit board in an image, obtaining the component positioning results. It can be understood that engineers collect circuit board images (including various types of components on the circuit board) and use these images to train a pre-built positioning model 101 until the positioning model 101 converges, obtaining a trained positioning model 101, which is then configured in the electronic device 100. The positioning model can employ any suitable neural network model; this application does not impose any specific limitations on it.

[0021] The host computer 200 includes any suitable type of device or apparatus, such as mobile devices, terminal devices, communication devices, computing devices, user equipment (UE), handheld devices, cloud devices, and servers. For example, in some embodiments, the host computer 200 can be a smartphone 201, a laptop computer 202, a desktop computer, a tablet computer, or an all-in-one computer. The host computer 200 is used to store the target image and image processing parameters. In practical applications, the host computer 200 transmits the target image and image processing parameters to the electronic device 100 via a network.

[0022] For example, in the embodiment of this application, the component positioning operation is performed as follows: First, the target image and image processing parameters input by the host computer 200 through the network are obtained.

[0023] Next, based on the image processing parameters, a cropped area image is obtained from the target image to obtain multiple candidate region images and the position information of each candidate region image in the target image.

[0024] Then, the candidate region image is input into a preset localization model for inference and localization to obtain the initial localization results of the elements in the candidate region image. The initial localization results include the initial coordinates of the top-left vertex and the bottom-right vertex of the initial bounding rectangle of the element.

[0025] Furthermore, the candidate bounding rectangles are integrated into the target image based on the width and height of the candidate region image and the position information of the candidate region image within the target image. The target image includes the candidate bounding rectangles, which are the initial bounding rectangles of the elements in the candidate region image.

[0026] Then, based on the image processing parameters and the candidate region image, the first and second bounding rectangles are determined in the target image.

[0027] Finally, based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, the candidate bounding rectangles are grouped to obtain multiple grouping results, where each grouping result includes at least one candidate bounding rectangle. The candidate bounding rectangles in the grouping results are then merged to obtain the final positioning result of the component.

[0028] The above methods enable effective correlation and fusion of cross-regional repeated detection results, reducing or avoiding problems such as repeated positioning, missed positioning, and boundary truncation caused by image block processing. This improves the integrity, accuracy, and consistency of component positioning results in large-size circuit board images, while also enhancing the reliability of component positioning and balancing the efficiency of large-size image processing.

[0029] It should be understood that Figure 1The application scenario shown is merely an illustrative representation of one instance of using electronic device 100 to locate elements in some embodiments of this application. Electronic device 100 is a server, but it does not impose any limitations on the structure, type, or quantity of electronic devices in any other embodiments.

[0030] To facilitate understanding of the component positioning method provided in the embodiments of this application, the electronic device provided in the embodiments of this application will first be described in detail.

[0031] Please see Figure 2 , Figure 2 The diagram shows a schematic representation of the structure of an electronic device provided in some embodiments of this application.

[0032] See Figure 2 As shown, the electronic device 100 includes at least one processor 110 and at least one memory 120 connected in communication. Figure 2 Taking a bus system 130, a processor 110, and a memory 120 as an example, the various components in the electronic device 100 are coupled together through the bus system 130, which is used to realize the connection and communication between the various components. It is easy to understand that the bus system 130 may include, in addition to the data bus, a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity and brevity, in... Figure 2 The various buses are all labeled as Bus System 130. This is understandable. Figure 2 The structures shown in the embodiments are merely illustrative and do not limit the structure of the electronic device described above. For example, the electronic device may also include components that are larger than... Figure 2 The structure shown has more or fewer components, or has the same as Figure 2 The diagram shows different configurations of the structure.

[0033] Specifically, processor 110 is configured to provide computational and control capabilities to support electronic device 100 in executing corresponding business logic and functions. For example, it supports electronic device 100 in executing the component positioning method provided in the embodiments of this application, or in executing the steps of any possible implementation of the component positioning method provided in the embodiments of this application. It is understood that processor 110 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or it can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0034] The memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, instructions, and modules, such as the program, instructions, and modules corresponding to the component positioning method in the embodiments of this application. In some embodiments, the memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system, an application program required for at least one function, and the data storage area may store data created according to the use of the processor 110. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the non-transitory software programs, instructions, and modules stored in the memory 120, thereby implementing the component positioning method provided in the embodiments of this application, or executing the steps in any possible implementation of the component positioning method provided in the embodiments of this application. In some embodiments, the memory 120 may include high-speed random access memory and may also include non-transitory memory. For example, at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 120 may also include memory remotely located relative to the processor 110, and these remotely located memories may be connected to the processor 110 through a communication network. It is understood that examples of the aforementioned communication networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0035] As can be understood from the above, the entity implementing the component positioning method provided in this application embodiment can be any suitable type of electronic device with certain computing and control capabilities, such as the aforementioned electronic device 100. In some feasible implementations, the component positioning method provided in this application embodiment can be implemented by a processor executing program instructions stored in a memory.

[0036] The component positioning method provided in this application will be described in detail below with reference to exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0037] For details, please refer to Figure 3 As shown, the component positioning method provided in this application embodiment includes steps S31 to S37 to achieve component positioning.

[0038] Step S31: Obtain the input target image and image processing parameters.

[0039] For example, the target image is a complete image of the circuit board to be inspected, including the components on the circuit board. Image processing parameters include a reference expansion length, the width and height of the target image, and the width and height of the reference region image. Understandably, engineers can customize the width and height of the reference region image and the reference expansion length according to the size of the target image and actual needs. The reference expansion length refers to the expansion length when cropping the candidate region image using the width and height of the reference region image as a reference. In this embodiment, the reference region image is also called the region of interest image.

[0040] For example, in response to a received component positioning command, a complete image of the circuit board to be inspected is acquired as the target image. In some embodiments, the target image is acquired by an industrial camera or is a complete image generated by stitching together multiple images to ensure that the target image covers the entire circuit board to be inspected.

[0041] The target image is parsed to read its attribute information, obtain its width and height, and the width is denoted as IMG_W and the height as IMG_H.

[0042] In this embodiment, pre-configured image processing parameters are read. The image processing parameters include at least the width of the reference region image (denoted as RoI_W), the height of the reference region image (denoted as RoI_H), and the reference extension length (denoted as EXT_L).

[0043] In some embodiments, the read image processing parameters are validated to determine whether the width RoI_W, height RoI_H, and reference extension length EXT_L of the reference region image meet preset constraints. If the constraints are met, the subsequent image cropping step is performed; if the constraints are not met, the corresponding parameters are replaced according to preset default parameters, or parameter configuration error information is output and the positioning process is terminated.

[0044] For example, the constraints include the width RoI_W of the reference region image being less than or equal to the width IMG_W of the target image, the height RoI_H of the reference region image being less than or equal to the height IMG_H of the target image, and the reference extension length EXT_L being greater than zero and less than the minimum of the width RoI_W and height RoI_H of the reference region image.

[0045] Step S32: Based on the image processing parameters, crop the target image to obtain multiple candidate region images and the position information of each candidate region image in the target image.

[0046] Among them, the width of the candidate region image is less than, equal to or greater than the width of the reference region image, and the height of the candidate region image is less than, equal to or greater than the height of the reference region image.

[0047] For example, the sum of the width RoI_W of the reference region image and the reference expansion length EXT_L is determined as the first width W1, and the sum of the height RoI_H of the reference region image and the reference expansion length EXT_L is determined as the first height H1. The width of the cropping window is set to the first width W1, and the height is set to the first height H1.

[0048] Please see Figure 4 In this embodiment of the application, a rectangular coordinate system XOY is constructed with the upper left corner of the target image 41 as the origin, and the coordinate system is established from the origin O. Initially, the cropping window 42 is moved sequentially along the first axis direction (i.e., the X-axis direction / horizontal axis direction) of the target image 41 in the first row (where only one cropping window exists at any given time). Figure 4Multiple cropping windows 42 (i.e., red dashed boxes are shown for illustrative purposes only) are used to crop candidate region images. The order of the candidate region images in the first axis direction and the order in the second axis direction (i.e., the Y-axis / vertical axis direction) of the target image 41 are used as the position information of the candidate region images in the target image. For example, in some embodiments, if the order of one candidate region image in the first axis direction of the target image 41 is 1 (i.e., in the first column) and the order in the second axis direction of the target image 41 is 2 (i.e., in the second row), then the position information of the candidate region image in the target image is: Y-2-X-1, indicating that the candidate region image is the first in the first axis direction and the second in the second axis direction.

[0049] For example, after cropping the first row along the first axis, the cropping window is moved to the coordinate axis. Move the cropping window to the top left corner of the cropping window at coordinates [coordinates missing]. At the overlapping position (where the cropping window is in the second row), continue to move the cropping window sequentially along the first axis in the second row, cropping out the candidate region image and determining the position information of the candidate region image in the target image. The number of rows (i.e., the order of the cropping windows along the second axis of the target image). .

[0050] Of course, it can also be from the origin O. Initially, the cropping window 42 is moved sequentially along the second axis of the target image 41 in the first column to crop candidate regions and determine their positions within the target image. After cropping along the second axis in the first column, the cropping window is moved to the coordinate... Move the cropping window to the top left corner of the cropping window at coordinates [coordinates missing]. At the overlapping position (where the cropping window is currently in the second column), the cropping window continues to move sequentially along the second axis in the second column, cropping out the candidate region image and determining the position information of the candidate region image within the target image. The column number (i.e., the order of the cropping windows along the first axis of the target image). .

[0051] During the moving cropping window process, when the distance between the right edge of the cropping window and the right edge of the target image is less than the first width W1, and the distance between the bottom edge of the cropping window and the bottom edge of the target image is greater than or equal to the first height H1, it indicates that the cropping window exceeds the right boundary of the target image but does not exceed the bottom boundary of the target image. The width of the cropping window is reduced to the distance between the right edge of the cropping window and the right edge of the target image, while the height of the cropping window remains unchanged at the first height H1. Then, the candidate region image is cropped with the cropping window after the width is reduced, and the position information of the candidate region image in the target image is determined.

[0052] When the distance between the right edge of the cropping window and the right edge of the target image is greater than or equal to the first width W1, and the distance between the bottom edge of the cropping window and the bottom edge of the target image is less than the first height H1, it indicates that the cropping window does not exceed the right boundary of the target image but exceeds the bottom boundary of the target image. The width of the cropping window is kept unchanged at the first width W1, and the height of the cropping window is reduced to the distance between the bottom edge of the cropping window and the bottom edge of the target image. Then, the candidate region image is cropped with the cropping window after the height reduction, and the position information of the candidate region image in the target image is determined.

[0053] When the distance between the right edge of the cropping window and the right edge of the target image is less than the first width W1, and the distance between the bottom edge of the cropping window and the bottom edge of the target image is less than the first height H1, it indicates that the cropping window exceeds both the right and bottom boundaries of the target image. The width of the cropping window is reduced to the distance between the right edge of the cropping window and the right edge of the target image, and the height of the cropping window is reduced to the distance between the bottom edge of the cropping window and the bottom edge of the target image. Then, the candidate region image is cropped using the cropping window with reduced width and height, and the position information of the candidate region image in the target image is determined.

[0054] Step S33: Input the candidate region image into the preset localization model for inference localization to obtain the initial localization result of the element in the candidate region image.

[0055] In this embodiment, the initial positioning result includes the initial coordinates of the top left corner vertex and the bottom right corner vertex of the initial bounding rectangle of the element.

[0056] For example, candidate region images are read sequentially, and the read candidate region images are input into a preset localization model as images to be inferred. The localization model is then used to infer and locate the elements in the candidate region images, and the initial localization results of the elements in the candidate region images are obtained.

[0057] In some embodiments, the candidate region image is preprocessed before model input, including adjusting the image size to the input size of the localization model, performing pixel value normalization, and converting the candidate region image into the data format corresponding to the localization model to meet the input requirements of the localization model.

[0058] For example, the preprocessed candidate region image is input into a pre-trained localization model for forward inference, resulting in multiple candidate detection results output by the localization model. Each candidate detection result includes the component category, target confidence, and the initial bounding rectangle of the component. The initial bounding rectangle includes the initial coordinates of the top-left vertex. and the initial coordinates of the lower right corner vertex The coordinates mentioned above are all determined with the top left corner of the candidate region image as the origin.

[0059] For example, it is determined whether there are still candidate region images that have not been reasoned. If they exist, the next candidate region image is read and input into the localization model for reasoning and localization to obtain the initial localization result of the element; if they do not exist, the reasoning and localization of the elements in all candidate region images is completed to obtain the initial localization result of the elements corresponding to all candidate region images.

[0060] Understandably, the localization model can employ object detection models based on convolutional neural networks or Transformer architectures, such as the YOLO series models, Faster R-CNN models, SSD models, or RT-DETR models. The localization model is pre-trained using training data containing a large number of circuit board component samples, enabling it to output the component category, target confidence, the initial bounding rectangle of the component, and the coordinates of the top-left and bottom-right vertices of the initial bounding rectangle. This ensures extremely high localization accuracy for different types of components.

[0061] Step S34: Integrate the candidate bounding rectangles into the target image based on the width and height of the candidate region image and the position information of the candidate region image in the target image.

[0062] The target image includes candidate bounding rectangles, which are the initial bounding rectangles of elements in the candidate region image.

[0063] For example, the width and height of the candidate region image are read, the initial bounding rectangle of each element in the candidate region image is obtained as the candidate bounding rectangle, and the initial coordinates of the top left vertex of the candidate bounding rectangle are read. and the initial coordinates of the lower right corner vertex Read the position information of the candidate region image in the target image (i.e., the order in the first axis direction and the order in the second axis direction of the target image).

[0064] For example, based on the position information of the candidate region image in the target image, the width and height of the candidate region image, and the width and height of the target image, a coordinate mapping relationship is established between the first coordinate system (i.e., the coordinate system under the candidate region image) and the second coordinate system (i.e., the coordinate system under the target image).

[0065] For each candidate bounding rectangle, based on the coordinate mapping relationship, the initial coordinates of the top-left vertex of the candidate bounding rectangle in the candidate region image are... and the initial coordinates of the lower right corner vertex The coordinates of the candidate bounding rectangle are transformed to the target image coordinate system to obtain the initial coordinates of the top-left and bottom-right vertices in the target image. It is understood that the transformed candidate bounding rectangle retains its original width and height; only the reference coordinate system is transformed.

[0066] For example, the transformed candidate bounding rectangle is added to the corresponding position in the target image, and the association information of the candidate bounding rectangle is retained. The association information includes the component category, the target confidence, and the initial coordinates of the top left and bottom right vertices of the candidate bounding rectangle.

[0067] After transforming the coordinates of all candidate bounding rectangles in all candidate region images and mapping them to the second coordinate system, all mapped candidate bounding rectangles are superimposed on the target image. The target image retains its original pixel data, only adding all candidate bounding rectangles and their annotation information to uniformly present the initial positioning results of elements in different candidate region images.

[0068] Step S35: Determine the first and second bounding rectangles in the target image based on the image processing parameters and the candidate region image.

[0069] In this embodiment, the first bounding rectangle is the bounding rectangle of the horizontally overlapping region, which is the region where adjacent candidate region images overlap each other in the first axis direction of the target image. The second bounding rectangle is the bounding rectangle of the vertically overlapping region, which is the region where adjacent candidate region images overlap each other in the second axis direction of the target image.

[0070] For example, candidate region images adjacent to each other along a first axis and along a second axis are determined according to the arrangement of candidate region images in the target image. Here, the first axis is the horizontal / X-axis direction of the target image, and the second axis is the vertical / Y-axis direction of the target image.

[0071] For example, for any two adjacent candidate region images along the first axis, the area where these two candidate region images overlap along the first axis is read as the horizontal overlapping region. Based on the height of the reference region image, the coordinates of the top-left vertex of the outer rectangle of the horizontal overlapping region are determined. Based on the width of the reference region image and the reference extension length, the coordinates of the bottom-right vertex of the outer rectangle of the horizontal overlapping region are determined. Based on the coordinates of the top-left and bottom-right vertices of the outer rectangle of the horizontal overlapping region, the first outer rectangle is constructed.

[0072] Similarly, for any two adjacent candidate region images along the second axis, the overlapping area of ​​these two candidate region images along the second axis is taken as the vertical overlapping region. Based on the width of the reference region image, the coordinates of the top-left vertex of the circumscribed rectangle of the vertical overlapping region are determined. Based on the height of the reference region image and the reference extension length, the coordinates of the bottom-right vertex of the circumscribed rectangle of the vertical overlapping region are determined. Based on the coordinates of the top-left and bottom-right vertices of the circumscribed rectangle of the vertical overlapping region, a second circumscribed rectangle is constructed.

[0073] It is understandable that the first and second bounding rectangles serve as the basis for subsequent judgments on whether candidate bounding rectangles are located in horizontal or vertical overlapping areas, thereby enabling the grouping and merging of candidate bounding rectangles.

[0074] In the above embodiments, based on image processing parameters and the positional relationship of each candidate region image in the target image, the bounding rectangles of all horizontally overlapping regions (i.e., the first bounding rectangle) and the bounding rectangles of all vertically overlapping regions (i.e., the second bounding rectangle) are pre-constructed. In this way, it is only necessary to determine the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles to quickly identify duplicate detection results caused by image overlap and cropping, thereby improving the efficiency of rectangle grouping and merging processing.

[0075] Step S36: Based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, group the candidate bounding rectangles to obtain multiple grouping results.

[0076] One grouping result includes at least one candidate bounding rectangle.

[0077] For example, all candidate bounding rectangles, the first bounding rectangle, and the second bounding rectangle in the target image are read, and the component category, target confidence, and initial coordinates of the top left and bottom right vertices corresponding to each candidate bounding rectangle are read.

[0078] For each candidate bounding rectangle, determine its positional relationship with the first and second bounding rectangles. Specifically, if the initial coordinates of the top-left or bottom-right vertex of the candidate bounding rectangle are within or on the boundary of the first bounding rectangle, then the candidate bounding rectangle intersects with the first bounding rectangle and is located in the horizontal overlapping region. If the initial coordinates of the top-left or bottom-right vertex of the candidate bounding rectangle are within or on the boundary of the second bounding rectangle, then the candidate bounding rectangle intersects with the second bounding rectangle and is located in the vertical overlapping region.

[0079] When the candidate bounding rectangle intersects with both the first and second bounding rectangles, it is determined that the candidate bounding rectangle is located in both the horizontal and vertical overlapping regions (i.e., the intersection of the horizontal and vertical overlapping regions). If the candidate bounding rectangle does not intersect with either the first or second bounding rectangle, it is determined that the candidate bounding rectangle is located in the non-overlapping region.

[0080] For example, based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, a region attribute label is assigned to each candidate bounding rectangle. The region attribute label includes non-overlapping region, horizontally overlapping region, vertically overlapping region, and horizontally and vertically intersecting overlapping region.

[0081] For example, for candidate bounding rectangles located in non-overlapping regions (i.e., candidate bounding rectangles with the region attribute label "non-overlapping region"), a non-overlapping group is established. This non-overlapping group includes all candidate bounding rectangles within the non-overlapping region. Since non-overlapping regions do not generate duplicate detections, there is no need to associate candidate bounding rectangles within non-overlapping regions with other candidate bounding rectangles.

[0082] For candidate bounding rectangles located within a horizontally overlapping region (i.e., candidate bounding rectangles with the region attribute label "horizontally overlapping region"), a horizontally overlapping group is created. This horizontally overlapping group includes all candidate bounding rectangles located within the horizontally overlapping region. All candidate bounding rectangles located within the same horizontally overlapping region and meeting preset matching conditions are searched, and these are added to the horizontally overlapping group. The preset matching conditions include one or more of the following: same component category, intersection-union ratio (IU) greater than an IU threshold, and center point distance less than a preset distance threshold. It should be understood that when multiple preset matching conditions are included, a candidate bounding rectangle must simultaneously meet all conditions to be considered as satisfying the preset matching conditions.

[0083] For candidate bounding rectangles located within a vertically overlapping region (i.e., candidate bounding rectangles with the region attribute label "vertical overlapping region"), create a vertically overlapping group. This vertically overlapping group includes all candidate bounding rectangles located within the vertically overlapping region. Search for all candidate bounding rectangles located within the same vertically overlapping region that meet preset matching conditions, and add all candidate bounding rectangles located within the same vertically overlapping region that meet the preset matching conditions to the vertically overlapping group.

[0084] For candidate bounding rectangles located in the horizontal and vertical overlapping areas (i.e., candidate bounding rectangles with the region attribute label "horizontal and vertical overlapping area"), create an overlapping cross-group. The vertical overlapping cross-group includes all candidate bounding rectangles located in the horizontal and vertical overlapping areas. Search for all candidate bounding rectangles located in the horizontal and vertical overlapping areas that meet the preset matching conditions, and add all candidate bounding rectangles located in the same horizontal and vertical overlapping area that meet the preset matching conditions to the overlapping cross-group.

[0085] Step S37: Merge the candidate bounding rectangles in the grouping results to obtain the final positioning result of the component.

[0086] In this embodiment, the final positioning result includes the final coordinates of the top left corner vertex and the bottom right corner vertex of the final bounding rectangle of the element.

[0087] For example, for each grouping result, this embodiment of the application obtains the association information of each candidate bounding rectangle in that grouping result. The association information includes the component category, target confidence level, and the initial coordinates of the top-left and bottom-right vertices of the candidate bounding rectangle.

[0088] Determine if the number of candidate bounding rectangles in the grouping result is 1. If the number of candidate bounding rectangles is 1, then directly determine the candidate bounding rectangle as the final bounding rectangle of the corresponding element, and generate the final positioning result of the corresponding element.

[0089] If the number of candidate bounding rectangles is not 1, determine whether the component categories corresponding to the multiple candidate bounding rectangles are the same. If the component categories corresponding to the multiple candidate bounding rectangles are different, select the component category with the highest target confidence as the component category corresponding to the grouping result. If the component categories corresponding to the multiple candidate bounding rectangles are the same, directly use that component category as the final category.

[0090] For example, the initial coordinates of the top-left and bottom-right vertices of all candidate bounding rectangles in the grouping result are obtained. The final bounding rectangle is determined based on all candidate bounding rectangles in the grouping result. For instance, the target confidence score corresponding to each candidate bounding rectangle is used as a weight to calculate the weighted average of the initial coordinates of the top-left vertices of all candidate bounding rectangles, resulting in the final coordinates of the top-left vertices of the element's final bounding rectangle; similarly, the weighted average of the initial coordinates of the bottom-right vertices of all candidate bounding rectangles is calculated to obtain the final coordinates of the bottom-right vertices of the element's final bounding rectangle.

[0091] In other embodiments, the initial coordinates of the top-left corner vertex of all candidate bounding rectangles are arithmetically averaged, and the initial coordinates of the bottom-right corner vertex of the candidate bounding rectangles are arithmetically averaged to obtain the final coordinates of the top-left corner vertex and the bottom-right corner vertex of the final bounding rectangle of the element.

[0092] In other embodiments, the minimum value of the x-coordinate and y-coordinate of the top-left corner vertex of all candidate bounding rectangles is taken as the final coordinate of the top-left corner vertex of the final bounding rectangle of the element, and the maximum value of the x-coordinate and y-coordinate of the bottom-right corner vertex of all candidate bounding rectangles is taken as the final coordinate of the bottom-right corner vertex of the final bounding rectangle of the element.

[0093] Finally, based on the determined final bounding rectangle, the final positioning results for the corresponding components are generated. The final positioning results include the component number, component category, the final coordinates of the top-left and bottom-right vertices of the final bounding rectangle, and the final confidence level.

[0094] For example, the output is the final localization result set. This set includes the final localization results of all elements in the target image, with each element corresponding to a final bounding rectangle, thus completing the localization of the elements in the target image.

[0095] This application embodiment locates the target image by cropping it into multiple candidate region images, and integrates the initial location results by combining the position information of the candidate region images in the target image. At the same time, it groups and merges the candidate bounding rectangles based on the positional relationship between the horizontal overlapping region, the vertical overlapping region and the candidate bounding rectangle, realizing the effective association and fusion of cross-regional repeated detection results. This reduces or even avoids the problems of repeated location, missed location and boundary truncation caused by image block processing, significantly improving the integrity, accuracy and consistency of component location results in large-size circuit board images. While taking into account the efficiency of large-size image processing, it improves the reliability of component location.

[0096] In some implementations, the embodiments of this application, through steps S321 to S325, achieve the following: based on image processing parameters, cropping a region of the target image to obtain multiple candidate region images and the position information of each candidate region image in the target image.

[0097] Step S321: Determine the first number of regular region images in the first axis direction of the target image based on the width of the target image and the width of the reference region image.

[0098] Step S322: Determine the second number of regular region images in the second axis direction of the target image based on the height of the target image and the height of the reference region image.

[0099] The width of the regular region image is greater than or equal to the width of the reference region image, and the height of the regular region image is greater than or equal to the height of the reference region image.

[0100] For example, the width of the target image is divided by the width of the reference region image to obtain a first number of regular region images in the first axis direction of the target image.

[0101] For example, dividing the height of the target image by the height of the reference region image yields a second number of regular region images in the second axis direction of the target image.

[0102] Step S323: Based on the image processing parameters, the first number of regular region images, and the second number of regular region images, determine the coordinates of the top left corner vertex and the bottom right corner vertex of the candidate region image to be cropped.

[0103] For example, based on image processing parameters, a first number of regular region images, and a second number of regular region images, the first and second axis coordinates of the top-left vertex of the candidate region image to be cropped are determined, and the first and second axis coordinates of the bottom-right vertex of the candidate region image to be cropped are determined. The coordinates of the top-left vertex include the first and second axis coordinates of the top-left vertex, and the coordinates of the bottom-right vertex include the first and second axis coordinates of the bottom-right vertex.

[0104] In some embodiments, this application embodiment uses steps S3231 to S3237 to determine the coordinates of the upper left corner vertex and the lower right corner vertex of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images.

[0105] Step S3231: In response to the fact that both the first quantity and the second quantity are greater than zero, calculate the first upper left coordinate based on the width of the reference region image and the order of the candidate region images in the first axis direction.

[0106] Step S3232: Calculate the second upper left coordinate based on the height of the reference region image and the order of the candidate region images in the second axis direction.

[0107] For example, the first top-left coordinate is the first axis coordinate of the top-left vertex of the candidate region image, and the second top-left coordinate is the second axis coordinate of the top-left vertex of the candidate region image. The coordinates of the top-left vertex of the candidate region image include the first top-left coordinate and the second top-left coordinate.

[0108] For example, in this embodiment of the application, the order of the candidate region image in the first axis direction of the target image is multiplied by the width of the reference region image to obtain the first upper-left coordinate, that is: , The first top-left coordinate, The order of candidate region images along the first axis of the target image. This indicates the width of the reference region image.

[0109] For example, in this embodiment of the application, the order of the candidate region image in the second axis direction of the target image is multiplied by the height of the reference region image to obtain the second upper-left coordinate, that is: , The second top-left coordinate, The order of candidate region images along the second axis of the target image. Indicates the height of the reference region image.

[0110] Step S3233: Calculate the first lower right initial coordinates based on the first upper left coordinates and the width of the reference region image.

[0111] Step S3234: Calculate the second lower right initial coordinates based on the second upper left coordinates and the height of the reference area image.

[0112] Wherein, the first lower-right initial coordinate is the first axis initial coordinate of the lower-right corner vertex of the candidate region image. The second lower-right initial coordinate is the second axis initial coordinate of the lower-right corner vertex of the candidate region image.

[0113] For example, in this embodiment of the application, the first upper-left coordinate is added to the width of the reference region image to obtain the first lower-right initial coordinate, that is: ,in, The first top-left coordinate, The width of the reference region image, The first initial coordinate is the bottom right corner.

[0114] For example, in this embodiment of the application, the second upper-left coordinate is added to the height of the reference region image to obtain the second lower-right initial coordinate, that is: ,in, The second top-left coordinate, The height of the reference region image, The second initial coordinate is the bottom right.

[0115] Step S3235: Determine the first expansion length and the second expansion length based on the first candidate distance, the second candidate distance, and the reference expansion length.

[0116] The first candidate distance is the distance between the right edge of the candidate region image and the right edge of the target image, and the second candidate distance is the distance between the bottom edge of the candidate region image and the bottom edge of the target image.

[0117] For example, the distance between the right edge of the candidate region image and the right edge of the target image is obtained as the first candidate distance, and the first candidate distance is compared with the reference expansion length. When the first candidate distance is less than the reference expansion length, the first candidate distance is determined as the first expansion length; when the first candidate distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the first expansion length.

[0118] For example, the distance between the lower edge of the candidate region image and the lower edge of the target image is obtained as the second candidate distance, and the second candidate distance is compared with the reference expansion length. When the second candidate distance is less than the reference expansion length, the second candidate distance is determined as the second expansion length; when the second candidate distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the second expansion length.

[0119] Step S3236: Add the first lower right initial coordinate to the first outer expansion length to obtain the first lower right final coordinate.

[0120] Step S3237: Add the second lower right initial coordinate to the second outer expansion length to obtain the second lower right final coordinate.

[0121] Wherein, the first bottom-right final coordinate is the first axis final coordinate of the bottom-right vertex of the candidate region image. The second bottom-right final coordinate is the second axis final coordinate of the bottom-right vertex of the candidate region image. The coordinates of the bottom-right vertex of the candidate region image include both the first and second bottom-right final coordinates.

[0122] For example, according to the following formula: Calculate the final coordinates of the first lower right corner, where, The final coordinates of the first lower right corner. The first initial coordinate is the bottom right corner. This is the first outward expansion length.

[0123] For example, according to the following formula: Calculate the final coordinates of the second lower right corner, where, The final coordinates are the second bottom right. The second initial coordinate is the bottom right. This is the second outward extension length.

[0124] In some embodiments, this application embodiment uses steps S323A to S323F to determine the coordinates of the upper left corner vertex and the lower right corner vertex of the candidate region image to be cropped, based on image processing parameters, a first number of regular region images, and a second number of regular region images.

[0125] Step S323A: In response to the first quantity being zero and the second quantity being greater than zero, multiply the first quantity by the width of the reference region image to obtain the first upper left coordinate.

[0126] Step S323B: Calculate the second upper left coordinate based on the height of the reference region image and the order of the candidate region images in the second axis direction.

[0127] For example, in this embodiment of the application, the first quantity is multiplied by the width of the reference region image to obtain the first upper-left coordinate, that is: , The first top-left coordinate, As the first quantity, The width of the reference region image.

[0128] For example, in this embodiment of the application, the order of the candidate region image in the second axis direction of the target image is multiplied by the height of the reference region image to obtain the second upper-left coordinate, that is: , The second top-left coordinate, The order of candidate region images along the second axis of the target image. Indicates the height of the reference region image.

[0129] Step S323C: Calculate the first lower right final coordinates based on the first upper left coordinates, the width of the reference region image, and the width of the target image.

[0130] Step S323D: Calculate the second lower right initial coordinates based on the second upper left coordinates and the height of the reference area image.

[0131] For example, in this embodiment of the application, the remainder of the target image width divided by the width of the reference region image (i.e., modulo operation) is obtained to get a first remainder; the first upper-left coordinate is added to the first remainder to obtain the first lower-right initial coordinate. That is: ,in, The final coordinates of the first lower right corner. The first top-left coordinate, The width of the target image. The width of the reference region image, To determine the width of the target image Divide by the width of the reference region image The remainder (i.e., the modulo operation).

[0132] For example, in this embodiment of the application, the second upper-left coordinate is added to the height of the reference region image to obtain the second lower-right initial coordinate, that is: ,in, The second top-left coordinate, The height of the reference region image, The second initial coordinate is the bottom right.

[0133] Step S323E: Determine the second expansion length based on the second candidate distance and the reference expansion length.

[0134] For example, the second candidate distance is compared with the reference expansion length. When the second candidate distance is less than the reference expansion length, the second candidate distance is determined as the second expansion length; when the second candidate distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the second expansion length.

[0135] Step S323F: Add the second lower right initial coordinate to the second outer expansion length to obtain the second lower right final coordinate.

[0136] For example, according to the following formula: Calculate the final coordinates of the second lower right corner, where, The final coordinates are the second bottom right. The second initial coordinate is the bottom right. This is the second outward extension length.

[0137] In some embodiments, this application embodiment uses steps S323G to S323L to determine the coordinates of the upper left corner vertex and the lower right corner vertex of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images.

[0138] Step S323G: In response to the first quantity being greater than zero and the second quantity being zero, calculate the first upper left coordinate based on the width of the reference region image and the order of the candidate region images along the first axis.

[0139] Step S323H: Multiply the second quantity by the height of the reference region image to obtain the second upper left coordinate.

[0140] For example, in this embodiment of the application, the order of the candidate region image in the first axis direction of the target image is multiplied by the width of the reference region image to obtain the first upper-left coordinate, that is: , The first top-left coordinate, The order of candidate region images along the first axis of the target image. This indicates the width of the reference region image.

[0141] For example, in this embodiment of the application, the second quantity is multiplied by the height of the reference region image to obtain the second upper-left coordinate, that is: , The second top-left coordinate, For the second quantity, The height of the reference region image.

[0142] Step S323I: Calculate the first lower right initial coordinates based on the first upper left coordinates and the width of the reference region image.

[0143] Step S323J: Calculate the second lower right final coordinates based on the second upper left coordinates, the height of the reference region image, and the height of the target image.

[0144] For example, in this embodiment of the application, the first upper-left coordinate is added to the width of the reference region image to obtain the first lower-right initial coordinate, that is: ,in, The first top-left coordinate, The width of the reference region image, The first initial coordinate is the bottom right corner.

[0145] For example, in this embodiment of the application, the remainder of the height of the target image divided by the height of the reference region image (i.e., modulo operation) is obtained to get a second remainder; the second upper-left coordinate is added to the second remainder to obtain the second lower-right final coordinate. That is: ,in, The final coordinates are the second bottom right. The second top-left coordinate, The height of the target image. The height of the reference region image, To extract the height of the target image Divide by the height of the reference region image The remainder (i.e., the modulo operation).

[0146] Step S323K: Determine the first expansion length based on the first candidate distance and the reference expansion length.

[0147] For example, the first candidate distance is compared with the reference expansion length. When the first candidate distance is less than the reference expansion length, the first candidate distance is determined as the first expansion length; when the first candidate distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the first expansion length.

[0148] Step S323L: Add the first lower right initial coordinate to the first outer expansion length to obtain the first lower right final coordinate.

[0149] For example, according to the following formula: Calculate the final coordinates of the first lower right corner, where, The final coordinates of the first lower right corner. The first initial coordinate is the bottom right corner. This is the first outward expansion length.

[0150] In some embodiments, this application embodiment uses steps S323M to S323Q to determine the coordinates of the upper left corner vertex and the lower right corner vertex of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images.

[0151] Step S323M: When both the first quantity and the second quantity are zero, multiply the first quantity by the width of the reference region image to obtain the first upper left coordinate.

[0152] Step S323N: Multiply the second quantity by the height of the reference region image to obtain the second upper left coordinate.

[0153] For example, in this embodiment of the application, the first quantity is multiplied by the width of the reference region image to obtain the first upper-left coordinate, that is: , The first top-left coordinate, As the first quantity, The width of the reference region image.

[0154] For example, in this embodiment of the application, the second quantity is multiplied by the height of the reference region image to obtain the second upper-left coordinate, that is: , The second top-left coordinate, For the second quantity, The height of the reference region image.

[0155] Step S323P: Calculate the first lower right final coordinates based on the first upper left coordinates, the width of the reference region image, and the width of the target image.

[0156] For example, in this embodiment of the application, the remainder of the target image width divided by the width of the reference region image (i.e., modulo operation) is obtained to get a first remainder; the first upper-left coordinate is added to the first remainder to obtain the first lower-right initial coordinate. That is: ,in, The final coordinates of the first lower right corner. The first top-left coordinate, The width of the target image. The width of the reference region image, To determine the width of the target image Divide by the width of the reference region image The remainder (i.e., the modulo operation).

[0157] Step S323Q: Calculate the second lower right final coordinates based on the second upper left coordinates, the height of the reference region image, and the height of the target image.

[0158] For example, in this embodiment of the application, the remainder of the height of the target image divided by the height of the reference region image (i.e., modulo operation) is obtained to get a second remainder; the second upper-left coordinate is added to the second remainder to obtain the second lower-right final coordinate. That is: ,in, The final coordinates are the second bottom right. The second top-left coordinate, The height of the target image. The height of the reference region image, To extract the height of the target image Divide by the height of the reference region image The remainder (i.e., the modulo operation).

[0159] Step S324: Based on the coordinates of the top left corner vertex and the bottom right corner vertex of the candidate region image, crop the candidate region image from the target image.

[0160] For example, in this embodiment of the application, based on the coordinates of the top-left and bottom-right vertices of the candidate region image, a first cropping coordinate (i.e., the coordinates of the top-left vertices of the candidate region image) and a second cropping coordinate (i.e., the coordinates of the bottom-right vertices of the candidate region image) are located in the target image. A rectangular cropping window is constructed based on the first and second cropping coordinates, and the candidate region image is obtained by cropping using the rectangular cropping window. It can be understood that the width of the rectangular cropping window is the difference between the first axis coordinate of the second cropping coordinate and the first axis coordinate of the first cropping coordinate, and the height of the rectangular cropping window is the difference between the second axis coordinate of the second cropping coordinate and the second axis coordinate of the first cropping coordinate.

[0161] Step S325: The order of the candidate region images in the first axis direction and the order in the second axis direction of the target image are determined as the position information of the candidate region images in the target image.

[0162] For example, if a candidate region image has an order of 2 along the first axis of the target image (i.e., in the second column) and an order of 1 along the second axis of the target image (i.e., in the first row), then the position information of this candidate region image in the target image is: Y-1-X-2, indicating that this candidate region image is the second one along the first axis and the first one along the second axis. It is understood that in this embodiment, the order along both the first and second axes is calculated starting from 0.

[0163] In some implementations, the embodiments of this application, through steps S351 to S356, determine a first bounding rectangle in the target image based on image processing parameters and candidate region images.

[0164] Step S351: Determine the first horizontal upper left coordinate as zero.

[0165] Step S352: Calculate the second horizontal upper left coordinate based on the height of the reference area image and the order of the horizontally overlapping areas in the second axis direction.

[0166] Wherein, the first horizontal top-left coordinate is the first-axis coordinate of the top-left vertex of the bounding rectangle of the horizontally overlapping region. The second horizontal top-left coordinate is the second-axis coordinate of the top-left vertex of the bounding rectangle of the horizontally overlapping region. It can be understood that the order of the horizontally overlapping regions in the second-axis direction refers to their order within the target image's second-axis direction after sorting all horizontally overlapping regions according to the target image's second-axis direction. The order of the horizontally overlapping regions in the second-axis direction is similar to the order of the candidate region images in the second-axis direction; the order of the horizontally overlapping regions in the second-axis direction is also calculated starting from 0.

[0167] For example, the first horizontal top-left coordinate is determined to be zero, that is: , The first horizontal coordinate is the top left coordinate.

[0168] For example, in this embodiment of the application, the order of the horizontally overlapping regions in the second axis direction is added to a preset value 1 to obtain a candidate order value. The height of the reference region image is multiplied by the candidate order value to obtain the second horizontal upper-left coordinate, that is: , The second horizontal coordinate is the top left. The order of the horizontally overlapping regions along the second axis. The candidate order value, Indicates the height of the reference region image.

[0169] Step S353: Determine the first actual expansion length based on the reference expansion length and the first reference distance.

[0170] In this embodiment, the first reference distance is the distance between the lower edge of the horizontally overlapping region and the lower edge of the target image.

[0171] For example, in this embodiment of the application, the distance between the lower edge of the horizontally overlapping region and the lower edge of the target image is obtained as a first reference distance, and the first reference distance is compared with a reference expansion length. If the first reference distance is less than the reference expansion length, the first reference distance is determined as the first actual expansion length; if the first reference distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the first actual expansion length.

[0172] Step S354: Determine the width of the target image as the first horizontal lower right coordinate.

[0173] Step S355: Add the second horizontal upper left coordinate to the first actual outward expansion length to obtain the second horizontal lower right coordinate.

[0174] Wherein, the first horizontal lower-right coordinate is the first axis coordinate of the lower-right corner vertex of the circumscribed rectangle of the horizontally overlapping region. The second horizontal lower-right coordinate is the second axis coordinate of the lower-right corner vertex of the circumscribed rectangle of the horizontally overlapping region.

[0175] For example, the first horizontal lower right coordinate is determined as the width of the target image, that is: , The first horizontal coordinate is the bottom right. Let be the width of the target image. Since it is a horizontally overlapping area, the first horizontal lower right coordinate is set to the width of the target image. In this way, the determined first bounding rectangle can cover the candidate bounding rectangles in the entire width direction of the target image.

[0176] For example, in this embodiment of the application, the second horizontal upper-left coordinate is added to the first actual outward expansion length to obtain the second horizontal lower-right coordinate, that is: , The second horizontal coordinate is the top left. This is the first actual outward expansion length. This is the second horizontal coordinate, located at the bottom right.

[0177] Step S356: Determine the first circumscribed rectangle based on the first horizontal upper left coordinate, the second horizontal upper left coordinate, the first horizontal lower right coordinate, and the second horizontal lower right coordinate.

[0178] For example, the first bounding rectangle is constructed based on the first axis coordinate (i.e., the first horizontal top-left coordinate) and the second axis coordinate (i.e., the second horizontal top-left coordinate) of the top-left vertex of the bounding rectangle of the horizontally overlapping region, and the first axis coordinate (i.e., the first horizontal bottom-right coordinate) and the second axis coordinate (i.e., the second horizontal bottom-right coordinate) of the bottom-right vertex of the bounding rectangle of the horizontally overlapping region.

[0179] In some implementations, the embodiments of this application, through steps S35A to S35F, determine a second circumscribed rectangle in the target image based on image processing parameters and candidate region images.

[0180] Step S35A: Calculate the first vertical upper left coordinate based on the width of the reference region image and the order of the vertically overlapping regions in the first axis direction.

[0181] Step S35B: Determine that the second vertical upper left coordinate is zero.

[0182] Wherein, the first vertical top-left coordinate is the first-axis coordinate of the top-left vertex of the circumscribed rectangle of the vertically overlapping region. The second vertical top-left coordinate is the second-axis coordinate of the top-left vertex of the circumscribed rectangle of the vertically overlapping region. It can be understood that the order of the vertically overlapping regions in the first-axis direction refers to their order within the first-axis direction of the target image after sorting all vertically overlapping regions according to the first-axis direction. The order of the vertically overlapping regions in the first-axis direction is similar to the order of the candidate region images in the first-axis direction; the order of the vertically overlapping regions in the first-axis direction is also calculated starting from 0.

[0183] For example, in this embodiment of the application, the order of the vertically overlapping regions in the first axis direction is added to a preset value 1 to obtain a reference order value. The width of the reference region image is multiplied by the reference order value to obtain the first vertical upper-left coordinate, that is: , The first vertical coordinate is the top left. The order of the vertically overlapping regions along the first axis. For reference order value, The width of the reference region image.

[0184] For example, the second vertical top-left coordinate is determined to be zero, that is: , This is the second vertical coordinate in the upper left corner.

[0185] Step S35C: Determine the second actual expansion length based on the reference expansion length and the second reference distance.

[0186] In this embodiment, the second reference distance is the distance between the right edge of the vertically overlapping region and the right edge of the target image.

[0187] For example, in this embodiment of the application, the distance between the right edge of the vertically overlapping region and the right edge of the target image is obtained as a second reference distance, and the second reference distance is compared with a reference expansion length. If the second reference distance is less than the reference expansion length, the second reference distance is determined as the second actual expansion length; if the second reference distance is greater than or equal to the reference expansion length, the reference expansion length is determined as the second actual expansion length.

[0188] Step S35D: Add the first vertical upper left coordinate to the second actual outward expansion length to obtain the first vertical lower right coordinate.

[0189] Step S35E: Determine the second vertical lower right coordinate as the height of the target image.

[0190] Wherein, the first vertical lower-right coordinate is the first axis coordinate of the lower-right corner vertex of the circumscribed rectangle of the vertically overlapping region. The second vertical lower-right coordinate is the second axis coordinate of the lower-right corner vertex of the circumscribed rectangle of the vertically overlapping region.

[0191] For example, in this embodiment of the application, the first vertical upper-left coordinate is added to the second actual outward expansion length to obtain the first vertical lower-right coordinate, that is: , The first vertical coordinate is the top left. This is the second actual outward expansion length. This is the first vertical coordinate, located to the lower right.

[0192] For example, the second vertical lower right coordinate is determined as the height of the target image, that is: , The second vertical coordinate is the bottom right. The height of the target image is set to the second vertical lower right coordinate. Since it is a vertically overlapping region, the height of the target image is set to the second vertical lower right coordinate. In this way, the determined second bounding rectangle can cover the candidate bounding rectangles in the entire height direction of the target image.

[0193] Step S35F: Determine the second circumscribed rectangle based on the first vertical upper left coordinate, the second vertical upper left coordinate, the first vertical lower right coordinate, and the second vertical lower right coordinate.

[0194] For example, a second circumscribed rectangle is constructed based on the first axis coordinate (i.e., the first vertical upper-left coordinate) and the second axis coordinate (i.e., the second vertical upper-left coordinate) of the top-left vertex of the circumscribed rectangle of the vertically overlapping region, and the first axis coordinate (i.e., the first vertical lower-right coordinate) and the second axis coordinate (i.e., the second vertical lower-right coordinate) of the bottom-right vertex of the circumscribed rectangle of the vertically overlapping region.

[0195] In some implementations, the embodiments of this application, through steps S361 to S363, group the candidate bounding rectangles according to the positional relationship between the candidate bounding rectangle and the first bounding rectangle and the second bounding rectangle, and obtain multiple grouping results.

[0196] Step S361: Construct a blank initial list of bounding rectangles.

[0197] Step S362: In response to the candidate bounding rectangle satisfying the first condition or the second condition, add the candidate bounding rectangle to the initial bounding rectangle list to obtain the final bounding rectangle list.

[0198] Among the image processing parameters is the cross-union ratio (CUR) threshold, which engineers can customize based on actual needs and experience data.

[0199] In this embodiment, the final list of bounding rectangles includes multiple candidate bounding rectangles to be grouped. The first condition is that the upper left corner vertex of a candidate bounding rectangle is located inside either the first or the second bounding rectangle, and the second condition is that the lower right corner vertex of a candidate bounding rectangle is located inside either the first or the second bounding rectangle.

[0200] It should be understood that the initial bounding rectangle list can be represented in any suitable form, such as an array or a queue, and the embodiments of this application do not impose any specific limitations on it.

[0201] For example, the top-left vertex of the candidate bounding rectangle is compared with the boundaries of the first and second bounding rectangles to determine whether the top-left vertex of the candidate bounding rectangle is located inside the first or second bounding rectangle. Specifically, if the top-left vertex of the candidate bounding rectangle is located inside or on the boundary of the first or second bounding rectangle, it is determined that the top-left vertex of the candidate bounding rectangle is located inside the first or second bounding rectangle, and the candidate bounding rectangle is added to the constructed initial bounding rectangle list.

[0202] For example, the lower right corner vertex of a candidate bounding rectangle is compared with the boundaries of the first and second bounding rectangles to determine whether the lower right corner vertex of the candidate bounding rectangle is located inside the first or second bounding rectangle. Specifically, if the lower right corner vertex of the candidate bounding rectangle is located inside or on the boundary of the first or second bounding rectangle, it is determined that the lower right corner vertex of the candidate bounding rectangle is located inside the first or second bounding rectangle, and the candidate bounding rectangle is added to the constructed initial bounding rectangle list. After comparing all candidate bounding rectangles, the final bounding rectangle list is obtained.

[0203] Step S363: Based on the intersection-union ratio threshold, group the candidate bounding rectangles in the final bounding rectangle list to obtain multiple grouping results.

[0204] Understandably, when the number of candidate bounding rectangles in the final bounding rectangle list is zero or one, that is, when there are no candidate bounding rectangles or only one candidate bounding rectangle in the final bounding rectangle list, there is no need to group the candidate bounding rectangles.

[0205] When the number of candidate bounding rectangles in the final bounding rectangle list is greater than or equal to two, that is, when there are two or more candidate bounding rectangles in the final bounding rectangle list, the candidate bounding rectangles in the final bounding rectangle list are grouped.

[0206] For example, multiple blank groups are constructed. The first candidate bounding rectangle in the final bounding rectangle list is selected and added to the first group. The remaining candidate bounding rectangles in the final bounding rectangle list are traversed. For each traversed candidate bounding rectangle, the intersection-union ratio (IUR) of the candidate bounding rectangle with each candidate bounding rectangle in the first group is calculated, and the calculated IUR is compared with a preset IUR threshold. When the IUR of the candidate bounding rectangle with any candidate bounding rectangle in the first group is greater than the IUR threshold, the candidate bounding rectangle is added to the first group. When the IUR of the candidate bounding rectangle with all candidate bounding rectangles in the first group is less than or equal to the IUR threshold, the candidate bounding rectangle is added to the second blank group.

[0207] Continue iterating through the remaining candidate bounding rectangles in the final bounding rectangle list. For each candidate bounding rectangle, calculate the intersection-union ratio (IUR) between that candidate bounding rectangle and each candidate bounding rectangle in the first group, and compare the calculated IUR with the IUR threshold. If the IUR between that candidate bounding rectangle and any candidate bounding rectangle in the first group is greater than the IUR threshold, add that candidate bounding rectangle to the first group. If the IUR between that candidate bounding rectangle and all candidate bounding rectangles in the first group is less than or equal to the IUR threshold, calculate the IUR between that candidate bounding rectangle and each candidate bounding rectangle in the second group, and compare the calculated IUR with the IUR threshold. When the intersection-union ratio (IUR) of the candidate bounding rectangle with any candidate bounding rectangle in the second group is greater than the IUR threshold, the candidate bounding rectangle is added to the second group; when the IUR of the candidate bounding rectangle with all candidate bounding rectangles in the second group is less than or equal to the IUR threshold, the candidate bounding rectangle is added to the third empty group.

[0208] This process continues, iterating through the remaining candidate bounding rectangles in the final bounding rectangle list. For each candidate bounding rectangle, the intersection-union ratio (IUR) is calculated between the IUR and every candidate bounding rectangle in all groups. The calculated IUR is then compared to an IUR threshold. Based on this comparison, the currently iterated candidate bounding rectangle is added to its corresponding group. This process is repeated until all candidate bounding rectangles in the final bounding rectangle list are grouped, resulting in multiple grouping results. Each grouping result must include at least one candidate bounding rectangle.

[0209] In some embodiments, the present application implements step S371 to merge the candidate circumscribed rectangles in the grouping results to obtain the final positioning result of the element.

[0210] Step S371: In response to the grouping result including a candidate bounding rectangle, combine the coordinates of the upper left and upper right corners of the candidate bounding rectangle, the initial foreground probability corresponding to the candidate bounding rectangle, and the initial type label to obtain the final positioning result of the element.

[0211] In this embodiment, the initial localization result also includes the initial foreground probability and initial type label corresponding to the candidate bounding rectangle. Correspondingly, the grouping result also includes the initial foreground probability and initial type label corresponding to the candidate bounding rectangle.

[0212] For example, for each grouping result, when the grouping result includes only one candidate bounding rectangle, the coordinates of the top-left and top-right vertices of the candidate bounding rectangle, the initial foreground probability corresponding to the candidate bounding rectangle, and the initial type label are combined to obtain the final positioning result of the element. It should be understood that the coordinates of the top-left vertices of the candidate bounding rectangle are the final coordinates of the top-left vertices of the final bounding rectangle of the element, and the coordinates of the bottom-right vertices of the candidate bounding rectangle are the final coordinates of the bottom-right vertices of the final bounding rectangle of the element.

[0213] In some embodiments, the present application embodiments achieve the merging of candidate circumscribed rectangles in the grouping results through steps S372 to S376 to obtain the final positioning result of the element.

[0214] Step S372: In response to the grouping result including multiple candidate bounding rectangles, the top-left coordinates of the first target and the top-left coordinates of the second target are respectively used as the first axis final coordinates and the second axis final coordinates of the top-left vertex of the final bounding rectangle of the element.

[0215] The top-left coordinate of the first target is the minimum value of the first-axis coordinates of the top-left vertices of the multiple candidate bounding rectangles. The top-left coordinate of the second target is the minimum value of the second-axis coordinates of the top-left vertices of the multiple candidate bounding rectangles. The final coordinates of the top-left vertex of the final bounding rectangle of the element include both the final first-axis coordinates and the final second-axis coordinates of the top-left vertex of the final bounding rectangle of the element.

[0216] For example, for each grouping result, when the grouping result includes multiple candidate bounding rectangles, the minimum value of the first axis coordinate of the top-left corner vertex of the multiple candidate bounding rectangles is selected as the first target top-left coordinate, and the minimum value of the second axis coordinate of the top-left corner vertex of the multiple candidate bounding rectangles is selected as the second target top-left coordinate. The first target top-left coordinate and the second target top-left coordinate are used as the first axis final coordinate and the second axis final coordinate of the top-left corner vertex of the final bounding rectangle of the element, respectively, to obtain the final coordinate of the top-left corner vertex of the final bounding rectangle of the element.

[0217] Step S373: Use the lower right coordinates of the first target and the lower right coordinates of the second target as the final coordinates of the first axis and the second axis of the lower right corner vertex of the final bounding rectangle of the element, respectively.

[0218] The bottom-right coordinate of the first target is the maximum value among the first-axis coordinates of the bottom-right corner vertices of multiple candidate bounding rectangles. The bottom-right coordinate of the second target is the maximum value among the second-axis coordinates of the bottom-right corner vertices of multiple candidate bounding rectangles. The final coordinates of the bottom-right corner vertex of the component's final bounding rectangle include both the final first-axis coordinate and the final second-axis coordinate of the bottom-right corner vertex of the component's final bounding rectangle.

[0219] For example, the maximum value among the first axis coordinates of the lower right corner vertices of multiple candidate bounding rectangles is selected as the lower right coordinate of the first target, and the maximum value among the second axis coordinates of the lower right corner vertices of multiple candidate bounding rectangles is selected as the lower right coordinate of the second target. The lower right coordinates of the first target and the lower right coordinates of the second target are used as the final first axis coordinates and the final second axis coordinates of the lower right corner vertices of the final bounding rectangle of the element, respectively, to obtain the final coordinates of the lower right corner vertices of the final bounding rectangle of the element.

[0220] Step S374: Determine the initial type label that appears most frequently in the grouping results as the final type label.

[0221] For example, in this embodiment of the application, the initial type label that appears most frequently among the initial type labels corresponding to all candidate bounding rectangles in the statistical grouping results is determined as the final type label.

[0222] Step S375: Determine the maximum value among one or more initial foreground probabilities corresponding to candidate bounding rectangles whose initial type label is the final type label in the grouping results as the final foreground probability.

[0223] For example, in this embodiment of the application, candidate bounding rectangles whose initial type label is the final type label are selected as reference bounding rectangles. It is understood that there may be one or more candidate bounding rectangles whose initial type label is the final type label, and therefore there may be one or more reference bounding rectangles. One or more initial foreground probabilities corresponding to the reference bounding rectangle are obtained, and the maximum value among the one or more initial foreground probabilities corresponding to the reference bounding rectangle is determined as the final foreground probability.

[0224] Step S376: Combine the final coordinates of the top left and bottom right vertices of the final bounding rectangle of the element, the final foreground probability, and the final type label to obtain the final positioning result of the element.

[0225] For example, the final coordinates of the top-left and bottom-right vertices of the final bounding rectangle of the combined element, the final foreground probability, and the final type label are used to obtain the final positioning result of the element.

[0226] The following is a brief example illustrating the implementation process of the embodiments of this application.

[0227] 1. Obtaining such as Figure 5 The target image 51 and image processing parameters are shown. The target image 51 includes various types of components 52 on the circuit board. The image processing parameters include the reference extension length, the width and height of the target image, and the width and height of the reference area image.

[0228] 2. See Figures 6 to 7C As shown, based on the image processing parameters, a cropped region image is obtained from the target image 51 to obtain multiple candidate region images 61 and the position information of each candidate region image 61 in the target image 51. Figure 6 In the middle, multiple candidate region images 61 include candidate region images y-0-x-0 (e.g., ... Figure 7A The image shown corresponds to the following location information: the zeroth candidate region image in the first axis direction, the zeroth candidate region image in the second axis direction, and the candidate region image y-0-x-1 (e.g., ...). Figure 7B The image shown corresponds to the following positional information: the first candidate region image along the first axis, the zeroth candidate region image along the second axis, candidate region image y-1-x-0, and candidate region image y-1-x-1 (e.g., ...). Figure 7C The image shown corresponds to the following location information: the first candidate region image in the first axis direction, the first candidate region image in the second axis direction, candidate region image y-2-x-0, and candidate region image y-2-x-1.

[0229] 3. Please refer to Figures 7A to 7C Each candidate region image 61 is input into a preset localization model for inference and localization, resulting in the initial localization result of the element in each candidate region image 61. It can be understood that the initial localization result includes the initial coordinates of the top-left vertex and the bottom-right vertex of the element's initial bounding rectangle. Figures 7A to 7C Rectangle 71 shown is the initial circumscribed rectangle (also known as the candidate circumscribed rectangle) of the component. Figures 7A to 7C The initial coordinates of the top-left and bottom-right vertices of the initial bounding rectangle of the component are not shown.

[0230] 4. Please refer to Figure 8A Based on the width and height of the candidate region image 61 and its position information within the target image 51, candidate bounding rectangles are integrated into the target image 51. The target image 51 includes one or more candidate bounding rectangles (i.e., Figure 8AThe rectangle 71 shown is the initial bounding rectangle of the element in the candidate region image 61.

[0231] 5. See Figure 8B As shown, based on the image processing parameters and candidate region images 61, a first bounding rectangle and a second bounding rectangle are determined in the target image 51. The first bounding rectangle is the bounding rectangle of the horizontally overlapping region, which is the region where adjacent candidate region images 61 overlap each other in the first axis direction of the target image 51. Figure 8B Rectangle 81 shown is the first circumscribed rectangle. The second circumscribed rectangle is the circumscribed rectangle of the vertically overlapping region, which is the area where adjacent candidate region images 61 overlap with each other in the second axis direction of the target image 51. Figure 8B Rectangle 82 shown in the diagram is the second circumscribed rectangle.

[0232] 6. Please refer to Figure 9 Based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, the candidate bounding rectangles are grouped to obtain multiple grouping results 91. Each grouping result 91 includes at least one candidate bounding rectangle.

[0233] 7. Please refer to Figure 10 The candidate bounding rectangles in the grouping results are merged to obtain the final positioning result of the component. The final positioning result includes the final coordinates of the top-left vertex and the bottom-right vertex of the component's final bounding rectangle. Figure 10 Rectangle 11 shown is the final circumscribed rectangle of the component. Figure 10 The final coordinates of the top-left and bottom-right vertices of the final bounding rectangle of the component are not shown.

[0234] In summary, the embodiments of this application locate the target image by cropping it into multiple candidate region images, integrate the initial positioning results by combining the positional information of the candidate region images in the target image, and group and merge the candidate bounding rectangles based on the positional relationship between the horizontal overlapping region, the vertical overlapping region and the candidate bounding rectangle. This achieves effective association and fusion of cross-regional repeated detection results, reduces or even avoids the problems of repeated positioning, missed positioning and boundary truncation caused by image block processing, significantly improves the integrity, accuracy and consistency of component positioning results in large-size circuit board images, and improves the reliability of component positioning while taking into account the efficiency of large-size image processing.

[0235] This application provides a computer-readable storage medium storing processor-executable program instructions. When executed by the processor, the program instructions cause the processor to perform the component positioning method provided in this application, or to perform the steps in any possible implementation of the component positioning method provided in this application.

[0236] In some embodiments, the storage medium may be a flash memory, a hard disk, an optical disk, a register, a magnetic surface memory, a removable disk, a CD-ROM, a random access memory (RAM), a read-only memory (ROM), an electrically programmable ROM, and an electrically erasable programmable ROM, or any other form of storage medium known in the art, or various devices including one or any combination of the above storage media.

[0237] In some embodiments, program instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0238] As an example, program instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, or in a single file dedicated to the program in question, or in multiple collaborative files (such as files that store one or more modules, subroutines, or code sections).

[0239] As an example, program instructions can be deployed to execute on a computing device (including devices such as smart terminals and servers), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network. It is understood that all or part of the steps of the methods described in the embodiments provided in this application can be implemented directly using electronic hardware or processor-executable program instructions, or a combination of both.

[0240] Those skilled in the art will understand that the embodiments provided in this application are merely illustrative. The order in which the steps in the methods of the embodiments are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged, and deleted according to actual needs. Modules or sub-modules, units or sub-units in the apparatus or system of the embodiments can be merged, divided, and deleted according to actual needs. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0241] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, and of course, it can also be implemented using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. It should be understood that the storage medium can be flash memory, hard disk, optical disk, register, magnetic surface memory, removable disk, CD-ROM, random access memory (RAM), read-only memory (ROM), electrically programmable ROM, and electrically erasable programmable ROM, etc.

[0242] It should be noted that the above embodiments are for illustrating the technical concept and features of this application, and are intended to enable those skilled in the art to understand the content of this application and implement it accordingly. They should not be construed as limiting the scope of protection of this application. Those skilled in the art can understand that all or part of the processes of the above embodiments can be implemented, modified according to the technical solutions described in the embodiments of this application, or equivalent substitutions can be made to some of the technical features. It is understood that these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should be considered as equivalent changes and modifications made based on the embodiments of this application, all of which should fall within the scope of the claims of this application.

Claims

1. A component positioning method, characterized in that, include: The target image and image processing parameters are obtained as input. The target image is a complete image of the circuit board to be inspected, including the components on the circuit board. The image processing parameters include the reference extension length, the width and height of the target image, and the width and height of the reference region image. Based on the image processing parameters, cropping regions of the target image are used to obtain multiple candidate region images and the position information of each candidate region image in the target image. The candidate region image is input into the preset localization model for inference localization, and the initial localization result of the element in the candidate region image is obtained. The initial localization result includes the initial coordinates of the upper left corner vertex and the lower right corner vertex of the initial bounding rectangle of the element. Based on the width and height of the candidate region image and the position information of the candidate region image in the target image, the candidate bounding rectangle is integrated into the target image. The target image includes the candidate bounding rectangle, which is the initial bounding rectangle of the element in the candidate region image. Based on the image processing parameters and candidate region images, a first bounding rectangle and a second bounding rectangle are determined in the target image. The first bounding rectangle is the bounding rectangle of the horizontally overlapping region, which is the region where adjacent candidate region images overlap with each other in the first axis direction of the target image. The second bounding rectangle is the bounding rectangle of the vertically overlapping region, which is the region where adjacent candidate region images overlap with each other in the second axis direction of the target image. Based on the positional relationship between the candidate bounding rectangle and the first and second bounding rectangles, the candidate bounding rectangles are grouped to obtain multiple grouping results. Each grouping result includes at least one candidate bounding rectangle. The candidate bounding rectangles in the grouping results are merged to obtain the final positioning result of the component. The final positioning result includes the final coordinates of the top left corner vertex and the bottom right corner vertex of the component's final bounding rectangle.

2. The component positioning method according to claim 1, characterized in that, The step of cropping a region from the target image based on image processing parameters to obtain multiple candidate region images and the position information of each candidate region image in the target image includes: Based on the width of the target image and the width of the reference region image, determine the first number of regular region images in the first axis direction of the target image; Based on the height of the target image and the height of the reference region image, determine the second number of regular region images in the second axis direction of the target image, wherein the width of the regular region image is greater than or equal to the width of the reference region image and the height of the regular region image is greater than or equal to the height of the reference region image. Based on the image processing parameters, the first number of regular region images, and the second number of regular region images, the coordinates of the top left vertex and the bottom right vertex of the candidate region image to be cropped are determined. Based on the coordinates of the top-left and bottom-right vertices of the candidate region image, the candidate region image is cropped from the target image. The order of the candidate region images in the first axis direction and the order in the second axis direction of the target image are determined as the position information of the candidate region images in the target image.

3. The component positioning method according to claim 2, characterized in that, The step of determining the coordinates of the top-left and bottom-right vertices of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images includes: If both the first and second quantities are greater than zero, calculate the first upper-left coordinate based on the width of the reference region image and the order of the candidate region images along the first axis. The first upper-left coordinate is the first axis coordinate of the upper-left vertex of the candidate region image. Based on the height of the reference region image and the order of the candidate region images in the second axis direction, the second upper left coordinate is calculated. The second upper left coordinate is the second axis coordinate of the upper left corner vertex of the candidate region image. The coordinates of the upper left corner vertex of the candidate region image include the first upper left coordinate and the second upper left coordinate. Based on the first upper left coordinate and the width of the reference region image, calculate the first lower right initial coordinate, which is the first axis initial coordinate of the lower right corner vertex of the candidate region image; Based on the second upper left coordinate and the height of the reference region image, calculate the second lower right initial coordinate, which is the second axis initial coordinate of the lower right vertex of the candidate region image; Based on the first candidate distance, the second candidate distance, and the reference expansion length, the first expansion length and the second expansion length are determined. The first candidate distance is the distance between the right edge of the candidate region image and the right edge of the target image, and the second candidate distance is the distance between the bottom edge of the candidate region image and the bottom edge of the target image. Add the first lower right initial coordinate to the first outer expansion length to obtain the first lower right final coordinate. The first lower right final coordinate is the first axis final coordinate of the lower right vertex of the candidate region image. The second lower right initial coordinate is added to the second outer expansion length to obtain the second lower right final coordinate. The second lower right final coordinate is the second axis final coordinate of the lower right vertex of the candidate region image. The coordinates of the lower right vertex of the candidate region image include the first lower right final coordinate and the second lower right final coordinate.

4. The component positioning method according to claim 3, characterized in that, The step of determining the coordinates of the top-left and bottom-right vertices of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images further includes: When the first quantity is zero and the second quantity is greater than zero, the first quantity is multiplied by the width of the reference region image to obtain the first top-left coordinate. The second upper-left coordinate is calculated based on the height of the reference region image and the order of the candidate region images along the second axis. Calculate the first lower right final coordinates based on the first upper left coordinates, the width of the reference region image, and the width of the target image; Calculate the initial coordinates of the second lower right region based on the second upper left coordinates and the height of the reference region image; The second expansion length is determined based on the second candidate distance and the reference expansion length; Add the second lower right initial coordinate to the second outer expansion length to obtain the second lower right final coordinate.

5. The component positioning method according to claim 3, characterized in that, The step of determining the coordinates of the top-left and bottom-right vertices of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images further includes: In response to a first quantity being greater than zero and a second quantity being zero, the first upper-left coordinate is calculated based on the width of the reference region image and the order of the candidate region images along the first axis. Multiply the second quantity by the height of the reference region image to obtain the second top-left coordinate; Calculate the first lower right initial coordinates based on the first upper left coordinates and the width of the reference region image; Calculate the final coordinates of the second lower right based on the second upper left coordinates, the height of the reference region image, and the height of the target image; The first expansion length is determined based on the first candidate distance and the reference expansion length; Add the first initial lower right coordinate to the first outer expansion length to obtain the first final lower right coordinate.

6. The component positioning method according to claim 3, characterized in that, The step of determining the coordinates of the top-left and bottom-right vertices of the candidate region image to be cropped based on image processing parameters, a first number of regular region images, and a second number of regular region images further includes: If both the first and second quantities are zero, the first quantity is multiplied by the width of the reference region image to obtain the first top-left coordinate. Multiply the second quantity by the height of the reference region image to obtain the second top-left coordinate; Calculate the first lower right final coordinates based on the first upper left coordinates, the width of the reference region image, and the width of the target image; The second lower right final coordinates are calculated based on the second upper left coordinates, the height of the reference region image, and the height of the target image.

7. The component positioning method according to claim 1, characterized in that, The step of determining the first bounding rectangle in the target image based on image processing parameters and candidate region images includes: The first horizontal top-left coordinate is set to zero. The first horizontal top-left coordinate is the first axis coordinate of the top-left vertex of the outer rectangle of the horizontal overlapping area. Based on the height of the reference region image and the order of the horizontally overlapping regions in the second axis direction, calculate the second horizontal upper left coordinate. The second horizontal upper left coordinate is the second axis coordinate of the upper left corner vertex of the outer rectangle of the horizontally overlapping region. The first actual expansion length is determined based on the reference expansion length and the first reference distance. The first reference distance is the distance between the lower edge of the horizontally overlapping area and the lower edge of the target image. The first horizontal bottom-right coordinate is determined as the width of the target image, and the first horizontal bottom-right coordinate is the first axis coordinate of the bottom-right vertex of the outer rectangle of the horizontal overlapping area; Add the second horizontal upper left coordinate to the first actual outward expansion length to obtain the second horizontal lower right coordinate. The second horizontal lower right coordinate is the second axis coordinate of the lower right corner vertex of the outer rectangle of the horizontal overlapping area. The first circumscribed rectangle is determined based on the first horizontal top-left coordinate, the second horizontal top-left coordinate, the first horizontal bottom-right coordinate, and the second horizontal bottom-right coordinate.

8. The component positioning method according to claim 1, characterized in that, The step of determining the second bounding rectangle in the target image based on image processing parameters and candidate region images includes: Based on the width of the reference region image and the order of the vertically overlapping regions in the first axis direction, calculate the first vertical upper left coordinate, which is the first axis coordinate of the upper left vertex of the outer rectangle of the vertically overlapping region; The second vertical top-left coordinate is set to zero. The second vertical top-left coordinate is the second axis coordinate of the top-left vertex of the outer rectangle of the vertical overlapping area. The second actual expansion length is determined based on the reference expansion length and the second reference distance. The second reference distance is the distance between the right edge of the vertical overlapping region and the right edge of the target image. Add the first vertical upper-left coordinate to the second actual outward expansion length to obtain the first vertical lower-right coordinate. The first vertical lower-right coordinate is the first axis coordinate of the lower-right corner vertex of the outer rectangle of the vertical overlapping area. The second vertical lower right coordinate is determined as the height of the target image, and the second vertical lower right coordinate is the second axis coordinate of the lower right corner vertex of the outer rectangle of the vertical overlapping area; The second circumscribed rectangle is determined based on the first vertical upper-left coordinate, the second vertical upper-left coordinate, the first vertical lower-right coordinate, and the second vertical lower-right coordinate.

9. The component positioning method according to claim 1, characterized in that, The image processing parameters also include an intersection-union (IU) threshold. The process involves grouping the candidate bounding rectangles according to their positional relationships with the first and second bounding rectangles, resulting in multiple grouping results, including: Construct a blank initial list of bounding rectangles; If a candidate bounding rectangle satisfies either the first or the second condition, it is added to the initial bounding rectangle list to obtain the final bounding rectangle list. The final bounding rectangle list includes multiple candidate bounding rectangles to be grouped. The first condition is that the upper left corner of the candidate bounding rectangle is located inside the first or the second bounding rectangle. The second condition is that the lower right corner of the candidate bounding rectangle is located inside the first or the second bounding rectangle. Based on the intersection-union ratio threshold, the candidate bounding rectangles in the final bounding rectangle list are grouped to obtain multiple grouping results.

10. The component positioning method according to claim 1, characterized in that, The grouping results also include the initial foreground probability and initial type label corresponding to the candidate bounding rectangles. Merging the candidate bounding rectangles in the grouping results to obtain the final positioning result of the element includes: In response to the grouping result including a candidate bounding rectangle, the coordinates of the top-left and top-right vertices of the candidate bounding rectangle, along with the initial foreground probability and initial type label corresponding to the candidate bounding rectangle, are combined to obtain the final positioning result of the element. The coordinates of the top-left vertices of the candidate bounding rectangle are the final coordinates of the top-left vertices of the element's final bounding rectangle, and the coordinates of the bottom-right vertices of the candidate bounding rectangle are the final coordinates of the bottom-right vertices of the element's final bounding rectangle; or, In response to the grouping result including multiple candidate bounding rectangles, the top-left coordinates of the first target and the top-left coordinates of the second target are respectively used as the final coordinates of the first axis and the second axis of the top-left vertex of the final bounding rectangle of the element. The top-left coordinate of the first target is the minimum value of the first axis coordinates of the top-left vertex of the multiple candidate bounding rectangles, and the top-left coordinate of the second target is the minimum value of the second axis coordinates of the top-left vertex of the multiple candidate bounding rectangles. The final coordinates of the top-left vertex of the final bounding rectangle of the element include the final coordinates of the first axis and the second axis of the top-left vertex of the final bounding rectangle of the element. The lower right coordinates of the first target and the lower right coordinates of the second target are respectively used as the final coordinates of the first axis and the second axis of the lower right corner vertex of the final bounding rectangle of the element. The lower right coordinate of the first target is the maximum value among the first axis coordinates of the lower right corner vertices of multiple candidate bounding rectangles, and the lower right coordinate of the second target is the maximum value among the second axis coordinates of the lower right corner vertices of multiple candidate bounding rectangles. The final coordinates of the lower right corner vertex of the final bounding rectangle of the element include the final coordinates of the first axis and the second axis of the lower right corner vertex of the final bounding rectangle of the element. The initial type label that appears most frequently in the grouping results is determined as the final type label; The maximum value among one or more initial foreground probabilities corresponding to candidate bounding rectangles whose initial type labels are the final type labels in the grouping results is determined as the final foreground probability. The final positioning result of the element is obtained by combining the final coordinates of the top left and bottom right vertices of the final bounding rectangle, the final foreground probability, and the final type label.

11. An electronic device, characterized in that, The device includes a processor and a memory, the processor being communicatively connected to the memory, the memory storing program instructions executable by the processor, the program instructions being executed by the processor to cause the electronic device to perform the component positioning method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores processor-executable program instructions, which, when executed by the processor, cause the processor to perform the component positioning method as described in any one of claims 1-10.

Images