Camera field of view assignment method, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of visual detection, and particularly relate to a camera field of view distribution method, an electronic device and a storage medium. According to the size and position relationship of the detection area rectangle, a tree structure relationship is constructed, and the first axis coordinate of the top left corner vertex of the detection area rectangle is extracted. In this way, when the candidate field of view is constructed according to the tree structure relationship, the first axis coordinate of the top left corner vertex of the candidate area rectangle and the size of the camera field of view, the processing time can be reduced, and the processing speed can be improved. Then, the multiple candidate fields of view are optimized and merged to obtain a field of view set. According to the field of view set, a field of view relationship is constructed, the field of view relationship is solved, and a field of view distribution result is obtained. In this way, the field of view distribution time is significantly reduced, and the field of view distribution efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to a camera field of view allocation method, electronic device and storage medium. Background Technology

[0002] When inspecting a PCB (Printed Circuit Board), a complete image of the PCB is typically obtained. Within this image, inspection areas are defined according to the inspection requirements. These inspection areas are small rectangles, smaller than the size of a camera's Field of Vision (FOV). Inspecting a PCB requires coverage by several FOVs. The goal of FOV allocation is to use the minimum number of FOVs while ensuring that each small rectangle is completely covered by at least one FOV. This is a combinatorial optimization problem, typically NP-complete. In practical applications, combinatorial optimization problems are difficult to solve optimally, and heuristic methods are generally used.

[0003] Among related technologies, the simplest heuristic method is the greedy algorithm, which is fast but usually only yields a limited number of solutions, typically one or two, and it's difficult to find the optimal solution among these. Another approach is based on linear programming, which can provide relatively optimized solutions. However, in cases where PCB boards have many defects, there are numerous small rectangular areas to inspect, potentially requiring dozens or even hundreds of FOVs, leading to long solution times and low efficiency for linear programming. Other heuristic methods (such as ant colony optimization and genetic algorithms) also suffer from excessively long solution times and low efficiency. For example, ant colony optimization requires multiple attempts and numerous iterations per attempt to ensure an optimal solution, resulting in excessively long solution times and low efficiency. Summary of the Invention

[0004] One objective of this application is to provide a camera field of view allocation method, electronic device, and storage medium to improve the long camera field of view allocation time and low efficiency in related technologies.

[0005] In a first aspect, embodiments of this application provide a camera field-of-view allocation method, comprising: acquiring an original image, wherein the original image is obtained by a camera capturing an image of a circuit board, the original image including a plurality of detection region rectangles, the size of the detection region rectangles being smaller than the size of the camera field of view; constructing a tree structure relationship based on the size and positional relationship of the detection region rectangles, the tree structure relationship being used to characterize the hierarchical relationship between a first reference region rectangle and the detection region rectangles and / or the hierarchical relationship between a second reference region rectangle and the first reference region rectangle, the first reference region rectangle including a plurality of detection region rectangles, the second reference rectangle including a plurality of first reference region rectangles; and extracting the left edge of all detection region rectangles. The first coordinate set is obtained by taking the first axis coordinate of the top-left vertex. The first coordinate set includes multiple distinct first axis coordinates. Based on the tree structure relationship, the first axis coordinate of the top-left vertex of the candidate region rectangle, and the size of the camera's field of view, candidate fields of view are constructed. The candidate region rectangle is any one of several detection region rectangles, and the candidate field of view covers one or more detection region rectangles. Multiple candidate fields of view are optimized and merged to obtain a field of view set, which includes one or more candidate fields of view. A field of view relationship is constructed based on the field of view set, and the field of view relationship is solved to obtain the field of view allocation result, which includes the target field of view and the position of the target field of view.

[0006] In some embodiments, the size of the camera's field of view includes the width of the camera's field of view. Constructing a candidate field of view based on the tree structure relationship, the first axis coordinate of the top-left vertex of the candidate region rectangle, and the size of the camera's field of view includes: obtaining reference coordinates of the top-left and bottom-right vertices of all detection region rectangles, the reference coordinates including first axis coordinates and second axis coordinates; determining the boundary coordinates of all detection region rectangles based on the reference coordinates of the top-left and bottom-right vertices of all detection region rectangles, the boundary coordinates including first edge coordinates and second edge coordinates, the first edge coordinate being the minimum value among the second axis coordinates of the top-left vertices of all detection region rectangles, and the second edge coordinate being the maximum value among the second axis coordinates of the bottom-right vertices of all detection region rectangles; and constructing a candidate field of view based on the boundary coordinates, the tree structure relationship, the width of the camera's field of view, and the first axis coordinate of the top-left vertices of the candidate region rectangles.

[0007] In some embodiments, the size of the camera field of view also includes the height of the camera field of view. Constructing a candidate field of view based on boundary coordinates, tree structure relationships, the width of the camera field of view, and the first axis coordinate of the top-left vertex of the candidate region rectangle includes: constructing a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate of the top-left vertex of the candidate region rectangle, and the width of the camera field of view; searching for a detection region rectangle located within the candidate rectangle as a reference rectangle based on the tree structure relationships; extracting the second axis coordinates of the top-left vertex of all reference rectangles to obtain a second coordinate set, which includes multiple distinct second axis coordinates; and constructing the candidate field of view based on the first axis coordinate of the top-left vertex of the candidate region rectangle, the second axis coordinates in the second coordinate set, and the width and height of the camera field of view.

[0008] In some embodiments, constructing a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate, and width of the camera's field of view of the top-left vertex of the candidate region rectangle includes: adding the difference between the width of the camera's field of view and a preset value to the first axis coordinate of the top-left vertex of the candidate region rectangle to obtain candidate coordinates; using the first axis coordinate and first edge coordinate of the top-left vertex of the candidate region rectangle as the first axis coordinate and second axis coordinate of the first vertex; using the candidate coordinates and second edge coordinate as the first axis coordinate and second axis coordinate of the second vertex; and constructing a candidate rectangle based on the first axis coordinate and second axis coordinate of the first vertex and the second axis coordinate of the second vertex, wherein the first vertex and the second vertex are the top-left vertex and the bottom-right vertex of the candidate rectangle, respectively.

[0009] In some embodiments, constructing a candidate field of view based on the first axis coordinate of the top-left vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, and the width and height of the camera's field of view includes: using the first axis coordinate and a first specified coordinate of the top-left vertex of the candidate region rectangle as the first axis coordinate and second axis coordinate of the third vertex, wherein the first specified coordinate is any one of the multiple second axis coordinates in the second coordinate set; adding the first axis coordinate of the top-left vertex of the candidate region rectangle to the difference between the width of the camera's field of view and a preset value to obtain a first reference coordinate; adding the first specified coordinate to the difference between the height of the camera's field of view and a preset value to obtain a second reference coordinate; using the first reference coordinate and the second reference coordinate as the first axis coordinate and second axis coordinate of the fourth vertex; and constructing a candidate field of view based on the first axis coordinate and second axis coordinate of the third vertex and the first axis coordinate and second axis coordinate of the fourth vertex, wherein the third vertex and the fourth vertex are the top-left vertex and the bottom-right vertex of the candidate field of view, respectively.

[0010] In some embodiments, optimizing and merging multiple candidate views to obtain a view set includes: in response to the number of candidate views being greater than or equal to a first number threshold, dividing the multiple candidate views into multiple datasets, each dataset including at least two candidate views; in response to the detection region rectangles covered by the first view being the same as those covered by the second view, deleting either the first view or the second view, where the first view and the second view are any two candidate views in the dataset; in response to the detection region rectangles covered by the first view being all covered by the second view, and some detection region rectangles covered by the second view not being covered by the first view, deleting the first view; in response to the detection region rectangles covered by the second view being all covered by the first view, and some detection region rectangles covered by the first view not being covered by the second view, deleting the second view; and in response to the candidate views in all datasets having been optimized and merged, optimizing and merging the optimized and merged candidate views in all datasets again to obtain a view set.

[0011] In some embodiments, the candidate views after optimization and merging in all datasets are further optimized and merged to obtain a view set, including: deleting either the third or fourth view if the detection region rectangles covered by the third view are the same as those covered by the fourth view, wherein the third view is any optimized and merged candidate view in the first dataset, and the fourth view is any optimized and merged candidate view in the second dataset, and the first and second datasets are any two datasets from multiple datasets; deleting the third view if all detection region rectangles covered by the third view are covered by the fourth view, and some detection region rectangles covered by the fourth view are not covered by the third view; deleting the fourth view if all detection region rectangles covered by the fourth view are covered by the third view, and some detection region rectangles covered by the third view are not covered by the fourth view; and deleting the view set if all candidate views after optimization and merging in all datasets have been further optimized and merged.

[0012] In some embodiments, a vision relationship is constructed based on the vision set, and the vision relationship is solved to obtain the vision allocation result, including: constructing a vision relationship based on the candidate visions of the coverage detection region rectangle; solving multiple vision relationships using a linear programming algorithm to obtain one or more target visions; and determining all target visions and their positions as the vision allocation result.

[0013] Secondly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the processor is communicatively connected to the memory, and the memory stores computer program instructions executable by the processor. When the computer program instructions are executed by the processor, the electronic device performs the camera field of view allocation method provided in the first aspect.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium storing processor-executable computer program instructions, which, when executed by a processor, cause the processor to perform the camera field-of-view allocation method provided in the first aspect.

[0015] The embodiments of this application have the following beneficial effects: Unlike related technologies, the embodiments of this application construct a tree structure relationship based on the size and positional relationship of the detection area rectangles, and extract the first axis coordinate of the top left corner vertex of all detection area rectangles. In this way, when constructing candidate fields of view based on the tree structure relationship, the first axis coordinate of the top left corner vertex of the candidate area rectangles, and the size of the camera field of view, the processing time can be reduced, the processing speed can be improved, multiple candidate fields of view can be optimized and merged to obtain a field of view set, the field of view relationship formula can be constructed based on the field of view set, and the field of view relationship formula can be solved to obtain the field of view allocation result. Thus, the field of view allocation time is reduced and the field of view allocation efficiency is improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the related technologies or embodiments will be briefly introduced below. Obviously, the drawings described below only show some embodiments of this application and should not be considered as limiting the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the application scenario of allocating camera field of view in some embodiments of this application; Figure 2 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application; Figure 3 This is a flowchart illustrating a camera field-of-view allocation method provided in some embodiments of this application; Figure 4 These are schematic diagrams of the original images in some embodiments of this application; Figure 5 This is a schematic diagram of the detection region rectangles and the constructed tree structure relationships in the original image provided in some embodiments of this application; Figure 6 yes Figure 3 A schematic diagram of a sub-process of step S34 in the camera field of view allocation method shown in the embodiment; Figure 7A This is an illustration of optimizing the merging of candidate views in some embodiments of this application. Figure 1 ; Figure 7BThis is an illustration of optimizing the merging of candidate views in some embodiments of this application. Figure 2 ; Figure 7C This is an illustration of optimizing the merging of candidate views in some embodiments of this application. Figure 3 ; Figure 8 This is an illustration of optimizing the merging of candidate views in some embodiments of this application. Figure 4 . Detailed Implementation

[0018] To make the objectives and advantages of the embodiments of this application more readily understood, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The detailed description of the embodiments of this application in the accompanying drawings is not intended to limit the scope of protection claimed by this application, but only represents selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that, unless there is a conflict, the various technical features involved in the embodiments of this application described below can be combined with each other, and all are within the protection scope of this application. Furthermore, although functional modules are divided in the device or structural schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," "third," and other similar expressions used herein do not limit the data or execution order, but are only for illustrative purposes and to distinguish identical or similar items with substantially the same function and effect, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features.

[0020] Unless otherwise defined, the technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. It should be understood that the term "and / or" as used in this specification includes any and all combinations of one or more of the listed items.

[0021] When inspecting a PCB (Printed Circuit Board), a complete image of the PCB is typically obtained. Within this image, inspection areas are defined according to the inspection requirements. These inspection areas are small rectangles, smaller than the size of a camera's Field of Vision (FOV). Inspecting a PCB requires coverage by several FOVs. The goal of FOV allocation is to use the minimum number of FOVs while ensuring that each small rectangle is completely covered by at least one FOV. This is a combinatorial optimization problem, typically NP-complete. In practical applications, combinatorial optimization problems are difficult to solve optimally, and heuristic methods are generally used.

[0022] Among related technologies, the simplest heuristic method is the greedy algorithm, which is fast but usually only yields a limited number of solutions, typically one or two, and it's difficult to find the optimal solution among these. Another approach is based on linear programming, which can provide relatively optimized solutions. However, in cases where PCB boards have many defects, there are numerous small rectangular areas to inspect, potentially requiring dozens or even hundreds of FOVs, leading to long solution times and low efficiency for linear programming. Other heuristic methods (such as ant colony optimization and genetic algorithms) also suffer from excessively long solution times and low efficiency. For example, ant colony optimization requires multiple attempts and numerous iterations per attempt to ensure an optimal solution, resulting in excessively long solution times and low efficiency.

[0023] In view of this, this application provides a camera field of view allocation method. A tree structure relationship is constructed based on the size and positional relationship of the detection region rectangles, and the first axis coordinate of the top left corner vertex of all detection region rectangles is extracted. In this way, when constructing candidate fields of view based on the tree structure relationship, the first axis coordinate of the top left corner vertex of the candidate region rectangles, and the size of the camera field of view, the processing time can be reduced and the processing speed can be improved. Then, multiple candidate fields of view are optimized and merged to obtain a field of view set. A field of view relationship is constructed based on the field of view set, and the field of view relationship is solved to obtain the field of view allocation result. In this way, the field of view allocation time is reduced and the field of view allocation efficiency is improved.

[0024] Please see Figure 1 , Figure 1 The illustration shows a schematic diagram of an application scenario for allocating camera field of view provided in some embodiments of this application.

[0025] See Figure 1 As shown, the application scenario includes an electronic device 100 and a camera 200. The electronic device 100 communicates with the camera 200 via a network. Examples of networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0026] For example, camera 200 is used to capture images of a circuit board to obtain an original image. The circuit board includes several component areas (typically formed by components on the circuit board). The original image includes several detection area rectangles corresponding to the several component areas, wherein the detection area rectangle is the smallest bounding rectangle of the component area, and the size of the detection area rectangle is smaller than the size of the camera's field of view.

[0027] The electronic device 100 is configured to execute the camera field of view allocation method provided in the embodiments of this application, specifically performing the camera field of view allocation operation: First, it acquires the original image captured by the camera 200 through the network.

[0028] Secondly, a tree structure relationship is constructed based on the size and positional relationship of the detection area rectangles. The tree structure relationship is used to characterize the hierarchical relationship between the first reference area rectangle and the detection area rectangle, and / or to characterize the hierarchical relationship between the second reference area rectangle and the first reference area rectangle.

[0029] Then, the first axis coordinates of the top left corner vertex of all detection region rectangles are extracted to obtain the first coordinate set, which includes multiple distinct first axis coordinates.

[0030] Then, based on the tree structure relationship, the first axis coordinate of the top left vertex of the candidate region rectangle, and the size of the camera field of view, a candidate field of view is constructed. The candidate region rectangle is any one of the several detection region rectangles.

[0031] Furthermore, multiple candidate views are optimized and merged to obtain a view set, which includes one or more candidate views.

[0032] Finally, a vision relationship is constructed based on the vision set, and the vision relationship is solved to obtain the vision allocation result, which includes the target vision and the position of the target vision.

[0033] The above methods significantly reduce camera field of view allocation time and improve camera field of view allocation efficiency.

[0034] It should be understood that Figure 1 The embodiments described herein are merely illustrative of one scenario in which an electronic device 100 is used to allocate the camera's field of view in some embodiments of this application. The electronic device 100 is a laptop computer, but it does not limit the structure, type, or number of electronic devices in other embodiments. For example, in some other embodiments, the electronic device may also be a desktop computer, tablet computer, microcontroller, FPGA chip, microcontroller, or other suitable type of device or component.

[0035] To facilitate understanding of the camera field of view allocation method provided in the embodiments of this application, the electronic device provided in the embodiments of this application will first be described in detail.

[0036] Please see Figure 2 , Figure 2 The diagram shows a schematic representation of the structure of an electronic device in some embodiments of this application.

[0037] See Figure 2 As shown, the electronic device 100 includes at least one processor 110 and at least one memory 120 connected in communication. Figure 2 Taking a bus system 130, a processor 110, and a memory 120 as an example, the various components in the electronic device 100 are coupled together through the bus system 130, which is used to realize the connection and communication between the various components. It is easy to understand that the bus system 130 may include, in addition to the data bus, a power bus, a control bus, and a status signal bus, etc., but for the sake of clarity and brevity, these will not be discussed further. Figure 2 All buses are labeled as Bus System 130. Understandably, Figure 2 The structures shown in the embodiments are merely illustrative and do not limit the structure of the electronic device described above. For example, the electronic device may also include components that are larger than... Figure 2 The structure shown has more or fewer components, or has the same as Figure 2 The diagram shows different configurations of the structure.

[0038] Specifically, processor 110 provides computational and control capabilities to support electronic device 100 in executing corresponding business logic and functions. For example, it supports electronic device 100 in executing the camera field of view allocation method provided in the embodiments of this application, or in executing the steps of any possible implementation of the camera field of view allocation method provided in the embodiments of this application. It is understood that processor 110 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or it can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0039] The memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, instructions, and modules, such as the program, instructions, and modules corresponding to the camera field-of-view allocation method in the embodiments of this application. In some embodiments, the memory 120 may include a program storage area and a data storage area. The program storage area stores the operating system and at least one application program required for a function, while the data storage area stores data created according to the use of the processor 110. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the non-transitory software programs, instructions, and modules stored in the memory 120, thereby implementing the camera field-of-view allocation method provided in the embodiments of this application, or executing the steps of any possible implementation of the camera field-of-view allocation method provided in the embodiments of this application. In some embodiments, the memory 120 may include high-speed random access memory and may also include non-transitory memory. For example, at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In other embodiments, the memory 120 may also include memories remotely located relative to the processor 110, which are connected to the processor 110 via a communication network. It is understood that examples of the aforementioned communication networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] As can be understood from the above, the camera field-of-view allocation method provided in this application embodiment can be implemented by any suitable type of electronic device with certain computing and control capabilities, such as the aforementioned electronic device 100. In some feasible implementations, the camera field-of-view allocation method provided in this application embodiment can be implemented by a processor executing computer program instructions stored in memory.

[0041] The camera field of view allocation method provided in this application will be described in detail below with reference to exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0042] Please see Figure 3 As shown, the camera field of view allocation method provided in this application embodiment includes steps S31 to S36 to achieve camera field of view allocation.

[0043] Step S31: Obtain the original image.

[0044] In this embodiment, the original image is obtained by a camera capturing images of the circuit board. The original image includes several detection area rectangles, each corresponding to a component area (typically formed by components on the circuit board). The component area can be a square, elliptical, circular, triangular, or other arbitrary shape. The detection area rectangle is the smallest bounding rectangle of the component area. The size of the detection area rectangle is smaller than the size of the camera's field of view.

[0045] For example, in embodiments of this application, the original image is acquired from the camera via a network, or the camera actively transmits the original image to an electronic device via a network.

[0046] In some embodiments, the original image may also be stored on any other suitable type of device or apparatus, such as a server or solid-state drive. In this application embodiment, the original image is obtained from the server or solid-state drive via a network.

[0047] For example, please see Figure 4 , Figure 4 The original images of some embodiments of this application are shown schematically. Figure 4 The green rectangle shown is the detection area rectangle 41, which is the component area. Figure 4 The smallest bounding rectangle (not shown).

[0048] Step S32: Construct a tree structure relationship based on the size and positional relationship of the detection area rectangle.

[0049] In this embodiment, the tree structure is used to characterize the hierarchical relationship between the first reference region rectangle and the detection region rectangle, and / or to characterize the hierarchical relationship between the second reference region rectangle and the first reference region rectangle. The first reference region rectangle includes multiple detection region rectangles, and the second reference region rectangle includes multiple first reference region rectangles.

[0050] Extract the region parameters for each detection region rectangle and establish a detection region rectangle parameter table. The region parameters include the coordinates of the top-left vertex, the coordinates of the bottom-right vertex, the width and height, and the coordinates of the center point.

[0051] For any two detection region rectangles, the positional relationship parameters of the two detection region rectangles are calculated based on their region parameters. These positional relationship parameters include the spacing along the first axis, the spacing along the second axis, the distance between the center points, the overlapping area, and the area of ​​the minimum circumscribed rectangle.

[0052] Based on the size and positional relationship parameters of the detection region rectangles, region clustering is performed on multiple detection region rectangles to obtain candidate cluster groups. For example, if any two detection region rectangles meet the following conditions: the distance between them along the first axis is less than a first distance threshold, the distance between them along the second axis is less than a second distance threshold, the distance between their center points is less than a third distance threshold, or one or more of the two detection region rectangles can be covered by the same camera's field of view, then the two detection region rectangles are determined to belong to the same candidate cluster group, ultimately resulting in multiple candidate cluster groups. Each candidate cluster group includes one or more detection region rectangles.

[0053] For each candidate cluster group, a corresponding first reference region rectangle is constructed. This is achieved by determining the smallest bounding rectangle of all detection region rectangles within the candidate cluster group, and this smallest bounding rectangle is designated as the first reference region rectangle. The first reference region rectangle covers all detection region rectangles within the corresponding candidate cluster group. A parent-child relationship is established between the first reference region rectangle as the parent node and the detection region rectangles as child nodes.

[0054] Extract the region parameters of each first reference region rectangle. For any two first reference region rectangles, calculate the spacing, overlap, and coverage relationship between the two first reference region rectangles based on their region parameters to obtain the positional relationship parameters of the two first reference region rectangles.

[0055] For example, based on the positional relationship parameters of the first reference region rectangles, region clustering is performed on multiple first reference region rectangles to obtain reference cluster groups. For instance, when any two first reference region rectangles meet the following conditions: the distance between the two first reference region rectangles is less than a fourth distance threshold, the size of the merged bounding rectangle does not exceed a preset size threshold, and they can be covered by one or more of the same camera's field of view, it is determined that the two first reference region rectangles belong to the same reference cluster group, and the two first reference region rectangles are merged into the same reference cluster group.

[0056] For each reference cluster group, a corresponding second reference region rectangle is constructed. This involves determining the smallest bounding rectangle of all first reference region rectangles within the reference cluster group, and defining this smallest bounding rectangle as the second reference region rectangle. The second reference region rectangle covers all first reference region rectangles within the corresponding reference cluster group. Using the second reference region rectangle as the parent node and the first reference region rectangles as child nodes, a parent-child relationship is established between the second and first reference region rectangles.

[0057] Finally, a tree structure is constructed based on the parent-child relationships between the first reference region rectangle and the detection region rectangle, and between the second reference region rectangle and the first reference region rectangle. The detection region rectangle serves as a leaf node, the first reference region rectangle as a first-level parent node (and also a child node of the second reference region rectangle), and the second reference region rectangle as a second-level parent node. This tree structure also characterizes the inclusion and aggregation relationships between the region rectangles.

[0058] By constructing a tree-like structure, when generating candidate fields of view, we can prioritize starting from the second reference region rectangle to determine whether it can be placed entirely into the field of view of a single camera. If not, we can decompose it down to the first reference region rectangle. If it still cannot be satisfied, we can continue to decompose it down to the detection region rectangle. This significantly reduces the number of candidate fields of view, providing a hierarchical search basis for subsequent "field of view optimization and merging" and "field of view relationship solving", reducing search time and thus improving the efficiency of camera field of view allocation.

[0059] For example, please see Figure 5 , Figure 5 a1 in the figure shows the detection region rectangle of the original image in some embodiments of this application. Figure 5 a2 in the text shows the result according to Figure 5 The tree structure relationship is constructed by the size and positional relationship of the rectangle of the detection area shown in a1. Figure 5 In rectangles a1 and a2, rectangles A through M are detection area rectangles; rectangles N, P, Q, R, and S are first reference area rectangles; and rectangles T and U are second reference area rectangles. Second reference area rectangle T includes three first reference area rectangles (i.e., first reference area rectangle N, first reference area rectangle P, and first reference area rectangle Q); second reference area rectangle U includes two first reference area rectangles (i.e., first reference area rectangle R and first reference area rectangle S). First reference area rectangle N includes two detection area rectangles (i.e., detection area rectangle A and detection area rectangle B); first reference area rectangle P includes three detection area rectangles (i.e., detection area rectangle C, detection area rectangle D, and detection area rectangle E); first reference area rectangle Q includes two detection area rectangles (i.e., detection area rectangle F and detection area rectangle G); first reference area rectangle R includes three detection area rectangles (i.e., detection area rectangle H, detection area rectangle I, and detection area rectangle J); and first reference area rectangle S includes three detection area rectangles (i.e., detection area rectangle K, detection area rectangle L, and detection area rectangle M).

[0060] Understandable Figure 5This is merely an illustrative representation of the hierarchical relationship of three levels of region rectangles. In other embodiments, a tree structure with more or fewer levels of region rectangles can be constructed. For example, a new region rectangle V can be constructed, which includes two second reference region rectangles (i.e., second reference region rectangle T and second reference region rectangle U). Alternatively, the new region rectangle V can be used as the second reference region rectangle, and the original second reference region rectangles T and U can be used as the first reference region rectangles. In this case, the new region rectangle V (i.e., the second reference region rectangle) includes two first reference region rectangles (i.e., the original second reference region rectangle T and second reference region rectangle U).

[0061] Step S33: Extract the first axis coordinates of the top left corner vertex of all detection region rectangles to obtain the first coordinate set.

[0062] In this embodiment, the first coordinate set includes multiple distinct first axis coordinates, meaning that any two first axis coordinates in the first coordinate set are different.

[0063] For example, in this embodiment of the application, an image coordinate system is established with the top left corner of the original image as the origin, where the horizontal direction is defined as the first axis and the vertical direction is defined as the second axis. All detection region rectangles in the original image are traversed, and the coordinate values ​​of the top left corner vertex of each detection region rectangle in the first axis direction are extracted to obtain the first axis coordinates of the top left corner vertex. Duplicate first axis coordinates are then removed, retaining only one corresponding first axis coordinate to obtain the first coordinate set.

[0064] The first axis is the X-axis of the image coordinate system of the original image. In some embodiments, the first axis may also be the Y-axis of the image coordinate system of the original image.

[0065] For example, for each first axis coordinate in the first coordinate set, a corresponding set of detection region rectangles is determined, and a coordinate mapping table is constructed based on the correspondence between the first axis coordinates and the sets of detection region rectangles. The coordinate mapping table records the first axis coordinate, the set of detection region rectangles corresponding to the first axis coordinate, and the number of detection region rectangles corresponding to the first axis coordinate. The set of detection region rectangles includes one or more detection region rectangles.

[0066] For example, the coordinate mapping table is shown in Table 1 below.

[0067] Table 1:

[0068] According to Table 1, detection region rectangles B1 and B2 have the same first axis coordinate. First axis coordinate The number of corresponding detection region rectangles is 2; the first axis coordinate of detection region rectangle C1 is... First axis coordinate The number of corresponding detection region rectangles is 1; detection region rectangles D1, D2, and D3 have the same first axis coordinate. First axis coordinate The number of corresponding detection region rectangles is 3.

[0069] Step S34: Construct candidate fields of view based on the tree structure relationship, the first axis coordinate of the top left vertex of the candidate region rectangle, and the size of the camera field of view.

[0070] In this embodiment, the candidate region rectangle is any one of several detection region rectangles, and the candidate field of view covers one or more detection region rectangles.

[0071] In this system, each first-axis coordinate in the first coordinate set serves as the candidate starting coordinate in the first-axis direction (i.e., the first-axis coordinate of the top-left vertex of the candidate region rectangle). Based on the candidate starting coordinates, the tree structure relationship, and the size of the camera's field of view, a candidate field of view is constructed. The position parameters of the candidate region rectangle itself are used to determine the starting position of the candidate field of view in the second-axis direction, while the tree structure relationship and the size of the camera's field of view guide the expansion of the candidate field of view and the determination of its coverage area.

[0072] For example, any one of the several detection region rectangles is selected as a candidate region rectangle, and the region parameters of the candidate region rectangle are extracted. The region parameters of the candidate region rectangle include the coordinates of the top-left vertex, width, and height, etc., where the coordinates of the top-left vertex include the first axis coordinate and the second axis coordinate of the top-left vertex.

[0073] For example, the top-left vertex of the candidate region rectangle is used as the initial positioning point of the candidate field of view, where the first axis coordinate and the second axis coordinate of the top-left vertex of the candidate region rectangle are the starting positions of the candidate field of view in the first axis direction and the second axis direction, respectively.

[0074] The camera's field of view includes both width and height. A field of view window is generated starting from the top-left corner of the candidate region rectangle, resulting in a candidate field of view rectangle. The left boundary of the candidate field of view rectangle is the first-axis coordinate of the top-left corner of the candidate region rectangle; the right boundary is the sum of the first-axis coordinate of the top-left corner of the candidate region rectangle and the width of the camera's field of view; the upper boundary is the second-axis coordinate of the top-left corner of the candidate region rectangle; and the lower boundary is the sum of the second-axis coordinate of the top-left corner of the candidate region rectangle and the height of the camera's field of view.

[0075] The set of coverage area rectangles associated with the candidate area rectangles is determined based on the tree structure relationship. That is, if the candidate area rectangle belongs to one of the first reference area rectangles, then all the detection area rectangles included in the first reference area rectangle are added to the set of coverage area rectangles; if the candidate area rectangle belongs to one of the second reference area rectangles, then all the descendant area rectangles (i.e., the first reference area rectangle and the detection area rectangle) included in the second reference area rectangle are added to the set of coverage area rectangles.

[0076] Coverage detection is performed on each region rectangle (including the second reference region rectangle, the first reference region rectangle, and / or the detection region rectangle) in the set of coverage region rectangles to determine whether the region rectangle is completely within the candidate field of view rectangle. If the left boundary of the region rectangle is greater than or equal to the left boundary of the candidate field of view rectangle, and the lower boundary of the region rectangle is greater than or equal to the lower boundary of the candidate field of view rectangle, and the right boundary of the region rectangle is less than or equal to the right boundary of the candidate field of view rectangle, and the upper boundary of the region rectangle is less than or equal to the upper boundary of the candidate field of view rectangle, then the region rectangle is determined to be within the candidate field of view rectangle (i.e., the candidate field of view covers the region rectangle), and is retained. If the left boundary of the region rectangle is less than the left boundary of the candidate field of view rectangle, and / or the lower boundary of the region rectangle is less than the lower boundary of the candidate field of view rectangle, and / or the right boundary of the region rectangle is greater than the right boundary of the candidate field of view rectangle, and / or the upper boundary of the region rectangle is greater than the upper boundary of the candidate field of view rectangle, then the region rectangle is determined to be outside the candidate field of view rectangle (i.e., the candidate field of view does not cover the region rectangle), and is discarded.

[0077] Finally, record the field of view information corresponding to the candidate field of view, including the position and size of the candidate field of view, the rectangle of the detection area covered, the identifiers and number of the first reference area rectangle and the second reference area rectangle, and the coverage area.

[0078] By analogy, the first axis coordinate of the top left corner vertex of each candidate region rectangle is taken as the candidate starting coordinate in the first axis direction. This process is repeated based on the candidate starting coordinate, tree structure relationship, and camera field of view size to eventually construct multiple candidate fields of view.

[0079] In this embodiment, the tree structure is used not only to store the hierarchical relationship of the region rectangles, but also to quickly determine and count the covered region rectangles, evaluate the coverage capability of candidate views, and reduce the amount of traversal calculations in the candidate view generation process.

[0080] Step S35: Optimize and merge multiple candidate views to obtain a view set.

[0081] In this embodiment, the field of view set includes one or more candidate fields of view. For example, the optimization objective of this embodiment is to minimize the number of candidate fields of view and the area of ​​overlapping coverage while ensuring that all detection region rectangles are covered.

[0082] For any candidate field of view, record the candidate field of view's number, location and size, the rectangle of the detection area it covers, the identifiers and quantities of the first reference area rectangle and the second reference area rectangle, and obtain the information table for that candidate field of view.

[0083] For any two candidate views, the number of rectangles in the area they jointly cover is calculated, and the coverage overlap is then calculated based on this number. Coverage overlap characterizes the degree of coverage similarity and redundancy between two candidate views.

[0084] For any two candidate fields of view, redundant candidate fields of view are identified based on the coverage overlap between the two candidate fields of view. That is, if the first candidate field of view is completely covered by the second candidate field of view, the first candidate field of view is determined to be a redundant candidate field of view, the first candidate field of view is removed, and the second candidate field of view is retained. In this way, the first optimization of candidate fields of view is completed.

[0085] After the first optimization of candidate fields of view, for any two remaining candidate fields of view, it is determined whether they meet the merging criteria. If the merging criteria are met, the minimum bounding rectangle of the two candidate fields of view is determined, and a new merged field of view with the same size as the minimum bounding rectangle is generated. The position, size, and covered area rectangle of the merged field of view are recorded. The merging criteria include: the size of the minimum bounding rectangle of the two candidate fields of view is less than or equal to the size of the camera's field of view; the two candidate fields of view correspond to the same first reference area rectangle; and the two candidate fields of view correspond to the same second reference area rectangle. If the merging criteria are not met, the candidate fields of view are not merged. In this way, the second optimization of candidate fields of view is completed, and finally one or more remaining candidate fields of view are obtained. These remaining one or more candidate fields of view are determined as a field of view set. The candidate fields of view set satisfy the following conditions: covering all detection area rectangles; no completely redundant candidate fields of view; minimum overlapping coverage area; and minimum number of candidate fields of view.

[0086] Step S36: Construct the field of view relation based on the field of view set, and solve the field of view relation to obtain the field of view allocation result.

[0087] In this embodiment, the field of view allocation result includes the target field of view and its location. The field of view relationship is used to characterize the coverage relationship between each candidate field of view in the field of view set and the detection region rectangle. Solving the field of view relationship determines the target field of view combination that can cover the entire detection region rectangle and has the fewest candidate field of view combinations.

[0088] For example, for any candidate field of view in the field of view set, record the candidate field of view's number, location, size, and the identifiers and number of the detection region rectangles it covers, thus obtaining information about the candidate field of view. Count all detection region rectangles, and construct a field of view coverage matrix based on the information of all candidate fields of view and all detection region rectangles.

[0089] For example, in some embodiments, the constructed field-of-view coverage matrix As shown in Table 2 below.

[0090] Table 2:

[0091] In Table 2, R1, R2, R3, R4, and R5 are the detection region rectangles. For the first A rectangular detection area, VF1, VF2, and VF3 are candidate fields of view. For the first One candidate field of view. View coverage matrix The element is denoted as ,when When, it indicates the candidate field of view. Coverage detection area rectangle ;when When, it indicates a candidate field of view. Uncovered detection area rectangle For example, as shown in Table 2, , indicating candidate field of view Coverage detection area rectangle ; , indicating candidate field of view Uncovered detection area rectangle .

[0092] For example, the embodiment of this application is the first... Candidate Vision Set selection variables To obtain the set of field of view selection variables ,in, , The number of candidate views. (Select variable) Used to indicate the first Whether a candidate viewpoint is selected. or . Indicates the first Candidate Vision Not selected Indicates the first Candidate Vision Selected.

[0093] For any given detection region rectangle, it is required that it be covered by at least one candidate field of view. Based on this, a coverage constraint relationship is constructed: Wherein, the covering constraint relationship represents the first Rectangular detection area At least by the Candidate Vision Coverage, the Rectangular detection area by Candidate Vision When covering, it indicates the first Candidate Vision Selected, i.e. .in, , This determines the number of rectangles in the detection area.

[0094] Construct the objective function ,in, , The objective function is used to characterize the number of selected candidate views. This means minimizing the number of candidate fields of view while ensuring coverage of all detection area rectangles. Among them, The function is a minimum value function.

[0095] Objective function Coverage constraint relationship The relationships are combined to form a field-of-view equation. A pre-defined optimization algorithm is used to solve the field-of-view equation, yielding the selection variables corresponding to all candidate fields of view. The value; based on the selected variable The value determines the target field of view (i.e. The corresponding selected number (1 candidate field of view); extract the parameters of the target field of view to generate the field of view allocation result. The parameters of the target field of view include the target field of view number, size, center coordinates, first axis coordinates and second axis coordinates of the top left vertex, and the identifier and number of the covered detection area rectangles, etc.

[0096] It should be understood that optimization algorithms include greedy covering algorithms, branch and bound algorithms, integer programming algorithms, dynamic programming algorithms, etc.

[0097] Through the above methods, the embodiments of this application can quickly generate candidate fields of view and optimize and merge the generated candidate fields of view, thereby efficiently and accurately solving the field of view relationship constructed based on the candidate fields of view, obtaining the field of view allocation result, significantly reducing the camera field of view allocation time, and improving the camera field of view allocation efficiency.

[0098] See Figure 6 As shown, in this embodiment of the application, through steps S341 to S343, a candidate field of view is constructed based on the tree structure relationship, the first axis coordinate of the top left corner vertex of the candidate region rectangle, and the size of the camera field of view.

[0099] Step S341: Obtain the reference coordinates of the top left and bottom right vertices of all detection region rectangles.

[0100] Step S342: Determine the boundary coordinates of all detection area rectangles based on the reference coordinates of the top left and bottom right vertices of all detection area rectangles.

[0101] In this embodiment, the reference coordinates include a first axis coordinate and a second axis coordinate. The boundary coordinates include a first edge coordinate and a second edge coordinate. The first edge coordinate is the minimum value of the second axis coordinates of the top-left vertices of all detection region rectangles, and the second edge coordinate is the maximum value of the second axis coordinates of the bottom-right vertices of all detection region rectangles.

[0102] For example, in this embodiment of the application, the reference coordinates (i.e., the first axis coordinates and the second axis coordinates) of the top left corner vertex of all detection area rectangles are extracted, and the reference coordinates (i.e., the first axis coordinates and the second axis coordinates) of the bottom right corner vertex of all detection area rectangles are extracted. The minimum value of the second axis coordinates of the top left corner vertex of all detection area rectangles is selected as the first edge coordinate, and the maximum value of the second axis coordinates of the bottom right corner vertex of all detection area rectangles is selected as the second edge coordinate, thus forming the boundary coordinates.

[0103] Step S343: Construct candidate views based on boundary coordinates, tree structure relationships, camera field of view width, and the first axis coordinate of the top left vertex of the candidate region rectangle.

[0104] In this embodiment, the size of the camera's field of view includes the width and height of the field of view. For example, the first vertex is determined based on the first axis coordinate of the top left corner vertex of the candidate region rectangle and the first edge coordinate in the boundary coordinates. The candidate coordinates are determined based on the first axis coordinate of the top left corner vertex of the candidate region rectangle and the width of the camera's field of view. The second vertex is determined based on the candidate coordinates and the second edge coordinate of the boundary coordinates. The candidate field of view is constructed based on the tree structure relationship, the first vertex, and the second vertex.

[0105] In some embodiments, the present application implements steps S3431 to S3434 to construct a candidate field of view based on the boundary coordinates, tree structure relationship, width of the camera field of view, and the first axis coordinate of the top left vertex of the candidate region rectangle.

[0106] Step S3431: Construct a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate of the top left corner vertex of the candidate region rectangle, and the width of the camera's field of view.

[0107] In some implementations, the embodiments of this application construct a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate, and width of the camera's field of view of the candidate region rectangle through steps S34311 to S34314.

[0108] Step S34311: Add the first axis coordinate of the top left corner vertex of the candidate region rectangle to the difference between the width of the camera's field of view and the preset value to obtain the candidate coordinates.

[0109] For example, embodiments of this application are based on the following formula: Calculate the candidate coordinates. In the above formula, Candidate coordinates, Let be the first-axis coordinate of the top-left vertex of the candidate region rectangle. For the width of the camera's field of view, The default value is 1, which means... .

[0110] Step S34312: Use the first axis coordinate and the first edge coordinate of the top left corner vertex of the candidate region rectangle as the first axis coordinate and the second axis coordinate of the first vertex.

[0111] Step S34313: Use the candidate coordinates and the second edge coordinates as the first axis coordinates and the second axis coordinates of the second vertex.

[0112] Step S34314: Construct candidate rectangles based on the first axis coordinates and second axis coordinates of the first vertex and the first axis coordinates and second axis coordinates of the second vertex.

[0113] In this embodiment, the first vertex and the second vertex are the top left vertex and the bottom right vertex of the candidate rectangle, respectively.

[0114] Let the first vertex (i.e., the top-left vertex of the candidate rectangle) be... Let the second vertex (i.e., the lower right vertex of the candidate rectangle) be... First vertex The first and second coordinate axes are the first coordinates of the top-left vertex of the candidate region rectangle. First edge coordinates That is, the first vertex The reference coordinates are Second vertex The first axis coordinate and the second axis coordinate are candidate coordinates. Second edge coordinates That is, the second vertex The reference coordinates are .

[0115] Based on the first and second axis coordinates of the first vertex and the first and second axis coordinates of the second vertex, a candidate rectangle is constructed. Specifically, a first straight line is drawn along the first axis (X-axis) from the first vertex, with a length equal to the absolute difference between the first and second axis coordinates of the first and second vertices. A second straight line is drawn along the second axis (Y-axis) from the first vertex, with a length equal to the absolute difference between the second and second axis coordinates of the first and second vertices. A third straight line is drawn along the opposite direction of the first axis (X-axis) from the second vertex, with a length equal to the absolute difference between the first and second axis coordinates of the first and second vertices. Finally, a fourth straight line is drawn along the opposite direction of the second axis (Y-axis) from the second vertex, with a length equal to the absolute difference between the second and second axis coordinates of the first and second vertices. Thus, the first, second, third, and fourth straight lines constitute the candidate rectangle.

[0116] Step S3432: Based on the tree structure relationship, search for the detection region rectangle located within the candidate rectangle as the reference rectangle.

[0117] For example, based on the tree structure, starting from the highest level, the system searches and determines whether the region rectangles (including the second reference region rectangle, the first reference region rectangle, and the detection region rectangle) at different levels are located within the candidate rectangle. When any second reference region rectangle is located within a candidate rectangle, the system determines the multiple first reference region rectangles included in that second reference region rectangle, and uses the detection region rectangle included in each first reference region rectangle as a reference rectangle.

[0118] Step S3433: Extract the second coordinate of the top left corner vertex of all reference rectangles to obtain the second coordinate set.

[0119] In this embodiment, the second coordinate set includes multiple distinct second axis coordinates, meaning that any two second axis coordinates in the second coordinate set are different.

[0120] For example, traverse all reference rectangles, extract the coordinates of the top-left vertex of all reference rectangles in the second axis direction, obtain the second axis coordinates of the top-left vertex, remove duplicate second axis coordinates, and keep only one corresponding second axis coordinate to obtain the second coordinate set.

[0121] Step S3434: Construct candidate fields of view based on the first axis coordinate of the top left corner vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, and the width and height of the camera's field of view.

[0122] For example, the third vertex is determined based on the first axis coordinate of the top left corner vertex of the candidate region rectangle and the second axis coordinate in the second coordinate set. The first reference coordinate and the second reference coordinate are determined based on the first axis coordinate of the top left corner vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, the width and height of the camera's field of view. The fourth vertex is determined based on the first reference coordinate and the second reference coordinate. The candidate field of view is constructed based on the third vertex and the fourth vertex.

[0123] In some implementations, the embodiments of this application construct a candidate field of view based on the first axis coordinate of the top left corner vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, and the width and height of the camera's field of view through steps S34341 to S34345.

[0124] Step S34341: Use the first axis coordinate and the first specified coordinate of the top left corner vertex of the candidate region rectangle as the first axis coordinate and the second axis coordinate of the third vertex.

[0125] Wherein, the first specified coordinate is any one of the multiple second-axis coordinates in the second coordinate set. For example, in this embodiment of the application, any one of the multiple second-axis coordinates in the second coordinate set is selected as the first specified coordinate. The first axis coordinate of the top left vertex of the candidate region rectangle. First specified coordinates The first and second axis coordinates of the third vertex.

[0126] Step S34342: Add the difference between the width of the camera's field of view and the preset value to the first axis coordinate of the top left corner vertex of the candidate region rectangle to obtain the first reference coordinate.

[0127] For example, embodiments of this application are based on the following formula: Calculate the first reference coordinates. In the above formula, As the first reference coordinate, Let be the first-axis coordinate of the top-left vertex of the candidate region rectangle. For the width of the camera's field of view, The default value is 1, which means... .

[0128] Step S34343: Add the difference between the height of the camera's field of view and the preset value to the first specified coordinates to obtain the second reference coordinates.

[0129] For example, embodiments of this application are based on the following formula: Calculate the second reference coordinates. In the formula, As the second reference coordinate, As the first specified coordinate, The height of the camera's field of view. The default value is 1, which means... .

[0130] Step S34344: Use the first reference coordinate and the second reference coordinate as the first axis coordinate and the second axis coordinate of the fourth vertex.

[0131] Step S34345: Construct candidate views based on the first and second axis coordinates of the third vertex and the first and second axis coordinates of the fourth vertex.

[0132] In this embodiment, the third vertex and the fourth vertex are the top left and bottom right vertices of the candidate field of view, respectively.

[0133] Let the third vertex (i.e., the top-left vertex of the candidate field of view) be... Let the fourth vertex (i.e., the lower right vertex of the candidate field of view) be... The third vertex The first and second coordinate axes are the first coordinates of the top-left vertex of the candidate region rectangle. First specified coordinates That is: the third vertex The reference coordinates are Among them, the fourth vertex The first axis coordinate and the second axis coordinate are respectively the first reference coordinate. Second reference coordinates That is, the fourth vertex The reference coordinates are .

[0134] Based on the first and second axis coordinates of the third vertex and the first and second axis coordinates of the fourth vertex, candidate views are constructed. Specifically, a fifth line is drawn along the first axis (X-axis) of the third vertex with a length equal to the absolute difference between the first axis coordinates of the third and fourth vertices. A sixth line is drawn along the second axis (Y-axis) of the third vertex with a length equal to the absolute difference between the second axis coordinates of the third and fourth vertices. A seventh line is drawn along the opposite direction of the first axis (X-axis) of the fourth vertex with a length equal to the absolute difference between the first axis coordinates of the third and fourth vertices. An eighth line is drawn along the opposite direction of the second axis (Y-axis) of the fourth vertex with a length equal to the absolute difference between the second axis coordinates of the third and fourth vertices. Thus, the fifth, sixth, seventh, and eighth lines constitute the candidate views.

[0135] In some embodiments, the present application implements steps S351 to S355 to optimize and merge multiple candidate views to obtain a view set.

[0136] Step S351: In response to the number of candidate views being greater than or equal to a first quantity threshold, the multiple candidate views are divided into multiple datasets.

[0137] In this embodiment, each dataset includes at least two candidate views. Engineers can customize the first threshold number based on their engineering experience and actual needs.

[0138] For example, when the number of candidate views is greater than or equal to the first number threshold, it indicates that there are too many candidate views. The multiple candidate views are divided into multiple datasets, and then the candidate views are optimized and merged.

[0139] In this application, multiple candidate fields of view can be divided into multiple datasets in any suitable manner. For example, multiple candidate fields of view can be divided into multiple datasets based on the spacing between candidate fields of view, the number of rectangles of the covered detection area, and the identifiers. This application does not impose any limitations on this.

[0140] Step S352: In response to the fact that the detection region rectangle covered by the first field of view is the same as the detection region rectangle covered by the second field of view, delete the first field of view or the second field of view.

[0141] In this embodiment, the first field of view and the second field of view are any two candidate fields of view in the dataset.

[0142] For example, for any dataset, this application embodiment selects any candidate field of view in the dataset as the first field of view, and selects any candidate field of view in the dataset other than the first field of view as the second field of view. The detection region rectangles covered by the first field of view and the detection region rectangles covered by the second field of view are compared. When the detection region rectangles covered by the first field of view and the detection region rectangles covered by the second field of view are consistent, it means that the first field of view and the second field of view are the same. Either the first field of view or the second field of view is deleted, that is, the first field of view or the second field of view is deleted.

[0143] For example, please see Figure 7A The first field of view 71 covers the detection area rectangles Z1 to Z4, and the second field of view 72 also covers the detection area rectangles Z1 to Z4. That is, the detection area rectangles covered by the first field of view 71 and the detection area rectangles covered by the second field of view 72 are the same. Delete the first field of view 71 or the second field of view 72.

[0144] Step S353: In response to the fact that all the detection area rectangles covered by the first field of view are covered by the second field of view, and that some of the detection area rectangles covered by the second field of view are not covered by the first field of view, delete the first field of view.

[0145] For example, when all the detection area rectangles covered by the first field of view are covered by the second field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) among the detection area rectangles covered by the second field of view that are not covered by the first field of view, it means that the second field of view has covered all the detection area rectangles covered by the first field of view. The first field of view is then deleted, and the second field of view is retained.

[0146] For example, please see Figure 7B The first field of view 71 covers the detection area rectangles Z1 to Z4, and the second field of view 72 covers the detection area rectangles Z1 to Z5. That is, all the detection area rectangles covered by the first field of view 71 are covered by the second field of view 72, and there is a detection area rectangle Z5 among the detection area rectangles Z1 to Z5 covered by the second field of view 72 that is not covered by the first field of view 71. The first field of view 71 is deleted, and the second field of view 72 is retained.

[0147] Step S354: In response to the fact that all the detection area rectangles covered by the second field of view are covered by the first field of view, and that some of the detection area rectangles covered by the first field of view are not covered by the second field of view, delete the second field of view.

[0148] For example, when all the detection area rectangles covered by the second field of view are covered by the first field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) among the detection area rectangles covered by the first field of view that are not covered by the second field of view, it means that the first field of view has covered all the detection area rectangles covered by the second field of view. The second field of view is then deleted, and the first field of view is retained.

[0149] For example, please see Figure 7C The first field of view 71 covers the detection area rectangles Z1 to Z5, and the second field of view 72 covers the detection area rectangles Z1 to Z4. That is, all the detection area rectangles covered by the second field of view 72 are covered by the first field of view 71, and there is a detection area rectangle Z5 among the detection area rectangles Z1 to Z5 covered by the first field of view 71 that is not covered by the second field of view 72. The second field of view 72 is deleted, and the first field of view 71 is retained.

[0150] Understandably, when there are some detection area rectangles (i.e., at least one detection area rectangle) in the detection area rectangles covered by the first field of view that are not covered by the second field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) in the detection area rectangles covered by the second field of view that are not covered by the first field of view, the first field of view and the second field of view are retained.

[0151] For example, please see Figure 8The first field of view 71 covers the detection area rectangles Z1 to Z4, and the second field of view 72 covers the detection area rectangles Z2 to Z5. That is, among the detection area rectangles Z1 to Z4 covered by the first field of view 71, there is a detection area rectangle Z1 that is not covered by the second field of view 72, and among the detection area rectangles Z2 to Z5 covered by the second field of view 72, there is a detection area rectangle Z5 that is not covered by the first field of view 71. The first field of view 71 and the second field of view 72 are retained.

[0152] Step S355: In response to the fact that the candidate views of all datasets have been optimized and merged, the candidate views of all datasets that have been optimized and merged are optimized and merged again to obtain the view set.

[0153] For example, after performing optimization and merging on the candidate views in all datasets, the optimized and merged candidate views in all datasets are optimized and merged again to obtain the view set.

[0154] In some implementations, the embodiments of this application use steps S3551 to S3554 to optimize and merge the candidate views after all data sets have been centrally optimized and merged, thereby obtaining a view set.

[0155] Step S3551: In response to the fact that the detection region rectangle covered by the third field of view is the same as the detection region rectangle covered by the fourth field of view, delete the third field of view or the fourth field of view.

[0156] The third view is any optimized and merged candidate view in the first dataset, the fourth view is any optimized and merged candidate view in the second dataset, and the first dataset and the second dataset are any two datasets from multiple datasets.

[0157] For example, in this embodiment of the application, any one of the multiple datasets is selected as the first dataset, and any one of the multiple datasets other than the first dataset is selected as the second dataset. Any optimized and merged candidate field of view in the first dataset is selected as the third field of view, and any optimized and merged candidate field of view in the second dataset is selected as the fourth field of view. The detection region rectangle covered by the third field of view is compared with the detection region rectangle covered by the fourth field of view. When the detection region rectangle covered by the third field of view is consistent with the detection region rectangle covered by the fourth field of view, it means that the third field of view and the fourth field of view are the same. Either the third field of view or the fourth field of view is deleted.

[0158] Step S3552: In response to the fact that all the detection area rectangles covered by the third field of view are covered by the fourth field of view, and that some of the detection area rectangles covered by the fourth field of view are not covered by the third field of view, delete the third field of view.

[0159] For example, when all the detection area rectangles covered by the third field of view are covered by the fourth field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) that are not covered by the third field of view, it means that the fourth field of view has covered all the detection area rectangles covered by the third field of view. The third field of view is then deleted, and the fourth field of view is retained.

[0160] Step S3553: In response to the fact that all the detection area rectangles covered by the fourth field of view are covered by the third field of view, and that some of the detection area rectangles covered by the third field of view are not covered by the fourth field of view, delete the fourth field of view.

[0161] For example, when all the detection area rectangles covered by the fourth field of view are covered by the third field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) among the detection area rectangles covered by the third field of view that are not covered by the fourth field of view, it means that the third field of view has covered all the detection area rectangles covered by the fourth field of view. The fourth field of view is then deleted, and the third field of view is retained.

[0162] Understandably, when there are some detection area rectangles (i.e., at least one detection area rectangle) in the detection area rectangles covered by the fourth field of view that are not covered by the third field of view, and there are some detection area rectangles (i.e., at least one detection area rectangle) in the detection area rectangles covered by the third field of view that are not covered by the fourth field of view, the third field of view and the fourth field of view are retained.

[0163] Step S3554: In response to the fact that all candidate views after optimization and merging in all datasets have been optimized and merged again, a view set is obtained.

[0164] After performing optimization and merging again on all candidate views after central optimization and merging of all datasets, it means that all candidate views after central optimization and merging of all datasets have been optimized and merged again, resulting in a view set, which includes one or more candidate views.

[0165] In some implementations, the embodiments of this application, through steps S361 to S363, construct a field-view relation based on the field-view set, solve the field-view relation, and obtain the field-view allocation result.

[0166] Step S361: Construct the field relationship formula based on the candidate fields of view of the coverage detection area rectangle.

[0167] Step S362: Use a linear programming algorithm to solve multiple vision relationship equations to obtain one or more target visions.

[0168] Step S363: Determine the field of view of all targets and the positions of all target fields of view as the field of view allocation result.

[0169] For example, each candidate field of view that covers the detection region rectangle in the field of view set is obtained, and the detection region rectangle covered by each candidate field of view is counted. The field of view relationship is constructed based on the detection region rectangle covered by each candidate field of view.

[0170] Specifically, for any given detection region rectangle, at least one candidate field of view must cover that detection region rectangle, thus constructing a corresponding field of view relationship. After constructing the corresponding field of view relationships for all candidate fields of view, the following multiple field of view relationships are obtained.

[0171]

[0172]

[0173] …

[0174]

[0175] Regarding the relationship between vision , , ... Representing candidate fields of view , ... Covered or uncovered detection area rectangle It should be understood that Indicates candidate field of view Coverage detection area rectangle , Indicates candidate field of view Uncovered detection area rectangle ; Indicates candidate field of view Coverage detection area rectangle , Indicates candidate field of view Uncovered detection area rectangle And so on, Indicates candidate field of view Coverage detection area rectangle , Indicates candidate field of view Uncovered detection area rectangle .

[0176] Similarly, for other formulas in the field of view relationship, the above formulas can be used as a basis. The meaning of each parameter in the formula, and the meaning of each parameter in other formulas, will not be elaborated here. It is easy to understand that the formula... That is, based on the rectangular detection area The corresponding constructed field-of-view relationship formula That is, based on the rectangular detection area The corresponding vision relationship obtained from the construction can be deduced by analogy, and so on, formula That is, based on the rectangular detection area The corresponding constructed field-of-view relationship formula To determine the rectangular detection area The corresponding constructed field-of-view relationship. Among them, This determines the number of rectangles in the detection area.

[0177] It should be understood that, based on the above multiple field-of-view relationships, the speed of solving multiple field-of-view relationships depends on the number of candidate fields of view and the number of detection region rectangles. Since the number of detection region rectangles is constant, it is necessary to minimize the number of candidate fields of view while covering all detection region rectangles. This will improve the solution speed, thereby reducing the processing time for camera field-of-view allocation and improving the efficiency of camera field-of-view allocation.

[0178] For example, in this embodiment of the application, a linear programming algorithm (e.g., integer programming algorithm, branch and bound algorithm, branch and cut plane algorithm, greedy cover algorithm) is used to solve multiple field-of-view relationships. The minimum number of target fields of view are determined from multiple candidate fields of view to cover the entire detection region rectangle, resulting in one or more target fields of view. All target fields of view and their positions are then determined as the field-of-view allocation result. The position of the target field of view includes the coordinates of its center point, the coordinates of its top-left vertex (including the coordinates of the first and second axes), and the coordinates of its bottom-right vertex (including the coordinates of the first and second axes).

[0179] In summary, the embodiments of this application construct a tree structure relationship based on the size and positional relationship of the detection region rectangles, and extract the first axis coordinate of the top left corner vertex of all detection region rectangles. In this way, when constructing candidate fields of view based on the tree structure relationship, the first axis coordinate of the top left corner vertex of the candidate region rectangles, and the size of the camera field of view, the processing time can be reduced, the processing speed can be improved, multiple candidate fields of view can be optimized and merged to obtain a field of view set, the field of view relationship formula can be constructed based on the field of view set, and the field of view relationship formula can be solved to obtain the field of view allocation result. Thus, the field of view allocation time is reduced and the field of view allocation efficiency is improved.

[0180] This application provides a computer-readable storage medium storing processor-executable computer program instructions. When executed by the processor, the computer program instructions cause the processor to perform the camera field-of-view allocation method provided in this application.

[0181] In some embodiments, the storage medium may be a flash memory, a hard disk, an optical disk, a register, a magnetic surface memory, a removable disk, a CD-ROM, a random access memory (RAM), a read-only memory (ROM), an electrically programmable ROM, and an electrically erasable programmable ROM, or any other form of storage medium known in the art, or various devices including one or any combination of the above storage media.

[0182] In some embodiments, computer program instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0183] As an example, computer program instructions may, but do not necessarily, correspond to files in a file system, and may be stored as part of a file that holds other programs or data, for example, in one or more scripts in an HTML (Hypertext Markup Language) document, or in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0184] As an example, computer program instructions can be deployed to execute on a single computing device (including devices such as smart terminals and servers), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network. It is readily understood that all or part of the steps of the methods described in the embodiments provided above can be implemented directly using electronic hardware or processor-executable computer program instructions, or a combination of both.

[0185] Those skilled in the art will understand that the embodiments provided in this application are merely illustrative. The order in which the steps in the methods of the embodiments are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged, and deleted according to actual needs. Modules or sub-modules, units or sub-units in the apparatus or system of the embodiments can be merged, divided, and deleted according to actual needs. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0186] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, and of course, it can also be implemented using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. It should be understood that the storage medium can be flash memory, hard disk, optical disk, register, magnetic surface memory, removable disk, CD-ROM, random access memory (RAM), read-only memory (ROM), electrically programmable ROM, and electrically erasable programmable ROM, etc.

[0187] It should be noted that the above embodiments are for illustrating the technical concept and features of this application, and are intended to enable those skilled in the art to understand the content of this application and implement it accordingly. They should not be construed as limiting the scope of protection of this application. Those skilled in the art can understand that all or part of the processes of the above embodiments can be implemented, modified according to the technical solutions described in the embodiments of this application, or equivalent substitutions can be made to some of the technical features. It is understood that these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should be considered as equivalent changes and modifications made based on the embodiments of this application, all of which should fall within the scope of the claims of this application.

Claims

1. A method for allocating camera field of view, characterized in that, include: The original image is obtained by the camera taking a picture of the circuit board. The original image includes several detection area rectangles, and the size of the detection area rectangles is smaller than the size of the camera's field of view. Based on the size and positional relationship of the detection area rectangles, a tree structure relationship is constructed. The tree structure relationship is used to characterize the hierarchical relationship between the first reference area rectangle and the detection area rectangle and / or the hierarchical relationship between the second reference area rectangle and the first reference area rectangle. The first reference area rectangle includes multiple detection area rectangles, and the second reference area rectangle includes multiple first reference area rectangles. Extract the first axis coordinates of the top left corner vertex of all detection region rectangles to obtain the first coordinate set, which includes multiple distinct first axis coordinates; Based on the tree structure relationship, the first axis coordinate of the top left vertex of the candidate region rectangle, and the size of the camera field of view, a candidate field of view is constructed. The candidate region rectangle is any one of several detection region rectangles, and the candidate field of view covers one or more detection region rectangles. Multiple candidate views are optimized and merged to obtain a view set, which includes one or more candidate views; Based on the set of fields of view, construct the field of view relation formula, solve the field of view relation formula, and obtain the field of view allocation result, which includes the target field of view and the position of the target field of view.

2. The camera field of view allocation method according to claim 1, characterized in that, The size of the camera's field of view includes its width. Constructing the candidate field of view based on the tree structure, the first axis coordinate of the top-left vertex of the candidate region rectangle, and the size of the camera's field of view includes: Obtain the reference coordinates of the top-left and bottom-right vertices of all detection region rectangles. The reference coordinates include the first axis coordinates and the second axis coordinates. Based on the reference coordinates of the top left and bottom right vertices of all detection area rectangles, determine the boundary coordinates of all detection area rectangles. The boundary coordinates include the first edge coordinates and the second edge coordinates. The first edge coordinates are the minimum value of the second axis coordinates of the top left vertices of all detection area rectangles, and the second edge coordinates are the maximum value of the second axis coordinates of the bottom right vertices of all detection area rectangles. Candidate views are constructed based on boundary coordinates, tree structure relationships, camera field of view width, and the first axis coordinate of the top-left vertex of the candidate region rectangle.

3. The camera field of view allocation method according to claim 2, characterized in that, The size of the camera's field of view also includes the height of the camera's field of view. The construction of the candidate field of view based on boundary coordinates, tree structure relationships, the width of the camera's field of view, and the first axis coordinate of the top-left vertex of the candidate region rectangle includes: Construct a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate of the top left vertex of the candidate region, and the width of the camera's field of view; Based on the tree structure relationship, the detection region rectangle located within the candidate rectangle is searched and used as the reference rectangle; Extract the second coordinates of the top-left corner vertex of all reference rectangles to obtain a second coordinate set, which includes multiple distinct second coordinates; Candidate views are constructed based on the first axis coordinate of the top-left vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, and the width and height of the camera's field of view.

4. The camera field of view allocation method according to claim 3, characterized in that, The process of constructing a candidate rectangle based on the first axis coordinate, first edge coordinate, second edge coordinate, and width of the camera's field of view of the top-left vertex of the candidate region rectangle includes: The candidate coordinates are obtained by adding the first axis coordinate of the top left corner vertex of the candidate region rectangle to the difference between the width of the camera's field of view and the preset value. Use the first axis coordinate and the first edge coordinate of the top left corner vertex of the candidate region rectangle as the first axis coordinate and the second axis coordinate of the first vertex; Use the candidate coordinates and the second edge coordinates as the first axis coordinates and the second axis coordinates of the second vertex; Based on the first and second axis coordinates of the first vertex and the first and second axis coordinates of the second vertex, construct a candidate rectangle, with the first vertex and the second vertex being the top left and bottom right vertex of the candidate rectangle, respectively.

5. The camera field of view allocation method according to claim 3, characterized in that, The process of constructing a candidate field of view based on the first axis coordinate of the top-left corner vertex of the candidate region rectangle, the second axis coordinate in the second coordinate set, and the width and height of the camera's field of view includes: The first coordinate and the first specified coordinate of the top left corner vertex of the candidate region rectangle are used as the first coordinate and the second coordinate of the third vertex. The first specified coordinate is any one of the multiple second coordinates in the second coordinate set. The first reference coordinate is obtained by adding the difference between the width of the camera's field of view and the preset value to the first axis coordinate of the top left corner vertex of the candidate region rectangle; Add the difference between the camera's field of view height and the preset value to the first specified coordinates to obtain the second reference coordinates; Use the first and second reference coordinates as the first and second axis coordinates of the fourth vertex; Candidate views are constructed based on the first and second axis coordinates of the third vertex and the first and second axis coordinates of the fourth vertex. The third and fourth vertices are the top left and bottom right vertices of the candidate views, respectively.

6. The camera field of view allocation method according to claim 1, characterized in that, The optimization and merging of multiple candidate views to obtain a view set includes: In response to the number of candidate views being greater than or equal to a first number threshold, the multiple candidate views are divided into multiple datasets, each dataset including at least two candidate views; If the rectangle of the detection region covered by the first field of view is the same as the rectangle of the detection region covered by the second field of view, delete the first field of view or the second field of view. The first field of view and the second field of view are any two candidate fields of view in the dataset. In response to the fact that all the detection area rectangles covered by the first field of view are covered by the second field of view, and that some of the detection area rectangles covered by the second field of view are not covered by the first field of view, the first field of view is deleted. In response to the fact that all the detection area rectangles covered by the second field of view are covered by the first field of view, and that some of the detection area rectangles covered by the first field of view are not covered by the second field of view, the second field of view is deleted. Since the candidate views of all datasets have been optimized and merged, the optimized and merged candidate views of all datasets are optimized and merged again to obtain the view set.

7. The camera field-of-view allocation method according to claim 6, characterized in that, The candidate views, after being optimized and merged from all datasets, are then optimized and merged again to obtain a view set, including: In response to the fact that the detection region rectangle covered by the third field of view is the same as the detection region rectangle covered by the fourth field of view, the third field of view or the fourth field of view is deleted. The third field of view is any optimized and merged candidate field of view in the first dataset, and the fourth field of view is any optimized and merged candidate field of view in the second dataset. The first dataset and the second dataset are any two datasets from multiple datasets. In response to the fact that all the detection area rectangles covered by the third field of view are covered by the fourth field of view, and that some of the detection area rectangles covered by the fourth field of view are not covered by the third field of view, the third field of view is deleted. In response to the fact that all detection area rectangles covered by the fourth field of view are covered by the third field of view, and that some detection area rectangles covered by the third field of view are not covered by the fourth field of view, the fourth field of view is deleted. In response to the optimization and merging of candidate views in all datasets, the set of views has been optimized and merged again.

8. The camera field-of-view allocation method according to any one of claims 1-7, characterized in that, The process of constructing a view relation based on the view set and solving the view relation to obtain the view allocation result includes: Based on the candidate fields of view of the covered detection area rectangle, construct the field of view relationship formula; A linear programming algorithm is used to solve multiple vision relationship equations to obtain one or more target visions. The field of view and the position of all target fields of view are determined as the field of view allocation result.

9. An electronic device, characterized in that, The device includes a processor and a memory, the processor being communicatively connected to the memory, the memory storing computer program instructions executable by the processor, and the computer program instructions, when executed by the processor, causing the electronic device to perform the camera field of view allocation method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores processor-executable computer program instructions, which, when executed by the processor, cause the processor to perform the camera field-of-view allocation method as described in any one of claims 1-8.

Images