Load Balancing Method, Device, Medium and Program Product for Multi-Camera System

Abstract

The present application discloses a load balancing method, device, medium and program product for a multi-camera system. Among them, the method determines the current region of interest of the target camera according to the pre-calibrated overlapping region between the cameras in the multi-camera system; obtains the current defect rate detected by the target camera within the target time period; determines the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; and adjusts the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera. In this technical solution, by monitoring the defect rate and load detected by each camera in real time, the region of interest of each camera is dynamically adjusted to achieve load balancing of the multi-camera system, improve the application performance and operation efficiency of the multi-camera system.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular, to a load balancing method, device, medium, and program product for a multi-camera system. Background Art

[0002] Due to advantages such as a wide coverage perspective range and high detection accuracy, multi-camera systems have been widely applied in many fields such as industrial production, intelligent transportation, medical image processing, and autonomous driving. When a multi-camera system is operating, in order to ensure that the multi-camera system can comprehensively cover the detection surface, there needs to be a certain pixel overlap area between adjacent cameras in the multi-camera system to achieve seamless connection and data fusion of the multi-camera system for the detection surface.

[0003] Currently, the processing methods for the pixel overlap area are mainly divided into two categories: based on feature points and based on the frequency domain. The former performs image stitching by extracting image feature points and matching corresponding point pairs; the latter achieves a more natural stitching effect by performing frequency domain analysis on the image. Although the above methods solve the problem of image stitching to a certain extent, when dealing with large-range image stitching in a complex environment, there are still problems such as unnatural stitching and low efficiency. In addition, the processing complexity of the pixel overlap area is relatively high, and it often has a large demand for computing resources, resulting in a load imbalance phenomenon in the multi-camera system. If the load of a certain camera in the multi-camera system is too high, it may lead to detection delay or missed detection of it, thus affecting the overall detection effect of the multi-camera system.

[0004] Therefore, how to provide a technical solution that can achieve intelligent collaborative operation of multi-cameras is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] The present application provides a load balancing method, device, medium, and program product for a multi-camera system. By real-time monitoring of the defect rate and load detected by each camera, dynamic adjustment of the region of interest of each camera is achieved, so that the multi-camera system reaches load balance, and the application performance and operation efficiency of the multi-camera system in industrial vision quality inspection are improved.

[0006] According to one aspect of the present application, a load balancing method for a multi-camera system is provided. The method includes:

[0007] Determine the current region of interest of the target camera according to the pre-calibrated overlap area between each camera in the multi-camera system; wherein, each camera in the multi-camera system is used to detect defects of the target product;

[0008] Obtain the current defect rate detected by the target camera within the target time period;

[0009] Determine the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera;

[0010] Adjust the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera.

[0011] According to another aspect of the present application, there is provided an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the load balancing method of the multi-camera system according to any embodiment of the present application.

[0012] According to another aspect of the present application, there is provided a computer-readable storage medium storing computer instructions for implementing the load balancing method of the multi-camera system according to any embodiment of the present application when executed by a processor.

[0013] According to another aspect of the present application, there is provided a computer program product including a computer program that implements the load balancing method of the multi-camera system according to any embodiment of the present application when executed by a processor.

[0014] The technical solution provided by the present application determines the current region of interest of the target camera according to the overlapping region calibrated in advance between the cameras in the multi-camera system; obtains the current defect rate detected by the target camera within the target time period; determines the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; and adjusts the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera. By monitoring the defect rate and load detected by each camera in real time, this technical solution realizes dynamic adjustment of the region of interest of each camera, enables the multi-camera system to achieve load balancing, and improves the application performance and operation efficiency of the multi-camera system in industrial vision quality inspection.

[0015] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0017] Figure 1 It is a flowchart of a load balancing method for a multi-camera system provided in Embodiment 1 of the present application.

[0018] Figure 2 It is a flowchart of a calibration method for an overlapping area provided in Embodiment 2 of the present application.

[0019] Figure 3 It is a schematic diagram of contour projection provided in Embodiment 2 of the present application.

[0020] Figure 4 It is a working schematic diagram of adjacent cameras provided in Embodiment 2 of the present application.

[0021] Figure 5 It is a contour projection diagram of the first camera provided in Embodiment 2 of the present application.

[0022] Figure 6 It is a contour projection diagram of the second camera provided in Embodiment 2 of the present application.

[0023] Figure 7 It is a schematic structural diagram of a device for implementing the load balancing method of a multi-camera system in the embodiments of the present application. Detailed implementation manners

[0024] In order to enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0025] It should be noted that the terms "target", "first", "second", "current", "reference", etc. in the specification, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1 FIG. is a flowchart of a load balancing method for a multi-camera system provided in Embodiment 1 of this application. This embodiment is applicable to the operation of a multi-camera system. This method can be executed by a load balancing device of the multi-camera system. The load balancing device of the multi-camera system can be implemented in the form of hardware and / or software, and the load balancing device of the multi-camera system can be configured in a device with data processing capabilities. As Figure 1 shown, the method includes the following steps.

[0028] S110. Determine the current region of interest of the target camera according to the pre-calibrated overlapping region between each camera in the multi-camera system. Among them, each camera in the multi-camera system is used to detect defects of the target product.

[0029] The multi-camera system can be a system composed of multiple intelligent cameras. For example, in the field of industrial production, when manufacturing semiconductors, PCBs, circuit boards or other precision devices, a multi-camera system can be used to perform large-area fine detection on products on the production line. Another example is that in the field of intelligent transportation, a multi-camera system can be used to implement functions such as road condition monitoring, vehicle identification, and traffic violation detection.

[0030] In this application, the multi-camera system can be used to detect defects of the target product. For example, in the semiconductor production process, the camera collects images of the wafers on the production line and identifies and detects defects in the wafer images, such as defects like scratches, bubbles, and cracks.

[0031] Since it is difficult for a single camera to cover a large detection area, in a multi-camera system, by combining multiple cameras and using the pixel overlapping regions between each camera, full-scene coverage and seamless detection can be achieved. In this application, the overlapping regions between each camera can be pre-calibrated according to the placement positions of each camera and / or the actual visible area range.

[0032] The current region of interest can be the region of concern determined in the image currently captured by the target camera according to the actual application scenario and task requirements. For example, the overlapping region range of two adjacent cameras is (2300, 3700). When the initial conditions of the two cameras are the same, the region of interest of the left camera can be set as (2300, 3000), and the region of interest of the right camera can be set as (3000, 3700).

[0033] S120. Obtain the current defect rate detected by the target camera within the target time period.

[0034] The current defect rate can be the defect rate obtained by the target camera for detecting the target product within the target time period. The higher the defect rate detected by the camera, the higher the power required for the camera to perform the defect recognition task; conversely, the lower it is. By statistically analyzing the current defect rate in this application, it is convenient to subsequently balance and adjust the load of each camera in the multi-camera system.

[0035] For example, the defect rate can be expressed as the ratio of the number of defective products to the total number of products, or can be expressed by the number of defects, or can also be expressed as the ratio of the defective area to the total area of all products.

[0036] Specifically, the image captured by the target camera within the target time period can be obtained first, then the number of defective products, defective area or number of defects in the captured image can be counted, and finally the defect rate detected by the target camera within the target time period can be determined according to the counted data.

[0037] Optionally, obtaining the current defect rate detected by the target camera within the target time period includes: obtaining the defective area and the total area detected by the target camera within the preset time period; determining the current defect rate detected by the target camera within the target time period according to the defective area and the total area.

[0038] Specifically, in this application, the defective area within the target time period is statistically analyzed in segments with a step size of stride. For example, the defective area detected by the target camera within the target time period is and the total area detected within the target time period is , then the current defect rate can be expressed by the following formula: .

[0039] In this application, the detection data of each camera in the multi-camera system can be periodically obtained by sampling, the defect rate of each time period can be calculated, and the corresponding histogram can be generated.

[0040] S130. Determine the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera.

[0041] Among them, the current load can be the CPU usage rate, memory occupancy rate, or the number of frames processed per second of the target camera, etc. For example, the CPU usage rate of the camera is used to represent the current load of the target camera.

[0042] In order to facilitate the accurate processing of the current defect rate and the current load, the current defect rate and the current load can be normalized respectively. For example, linear normalization, standard normalization, logarithmic function normalization, arctangent function normalization, etc. can be used to normalize the current defect rate and the current load.

[0043] Optionally, before determining the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera, the method further includes: obtaining the historical defect rates of the target camera in a plurality of historical time periods, and determining the maximum defect rate and the minimum defect rate in the historical defect rates; normalizing the current defect rate according to the maximum defect rate and the minimum defect rate to determine the current standard defect rate.

[0044] In this application, by normalizing the current defect rate, the error impact caused by the difference in the total detection area in different time periods on the defect rate is reduced, thereby improving the accuracy of subsequent load balancing adjustment.

[0045] Specifically, in this application, the linear normalization method is used to control the current defect rate between 0 and 1. The current standard defect rate can be expressed by the following formula:

[0046] ;

[0047] In the formula, represents the current standard defect rate, D represents the current defect rate, represents the minimum defect rate in the historical defect rates, represents the maximum defect rate in the historical defect rates.

[0048] Furthermore, determine the current load evaluation value of the target camera according to the current defect rate and the current load. Among them, the load evaluation value is used to represent the load level of the camera. The higher the load evaluation value, the greater the load of the camera, and the camera performance will also decline relatively.

[0049] In this application, the weighted sum of the normalized current defect rate and the current load can be obtained to get the current load evaluation value of the target camera; or the current defect rate and the current load can be multiplied to get the current load evaluation value of the target camera.

[0050] S140. Adjust the current region of interest of the target camera according to the current load evaluation value, and determine the target region of interest of the target camera.

[0051] In this application, if the current load evaluation value of the target camera is higher than a certain threshold, the current region of interest of the target camera can be reduced. For example, all or part of the region in the current region of interest of the target camera can be allocated to adjacent cameras; if the current load evaluation value of the target camera is lower than a certain threshold, the current region of interest of the target camera can be increased. For example, part of the region of interest of adjacent cameras can be taken over; if the current load evaluation value of the target camera is within a certain range, the current region of interest of the target camera can remain unchanged.

[0052] Specifically, it can be dynamically adjusted according to the current load evaluation value and the actual requirements of the scene. For example, the adjustment is carried out through the following formula: ;

[0053] In the formula, represents the target region of interest, represents the current region of interest, and are adjustment parameters and can be dynamically adjusted according to the actual requirements of the scene.

[0054] Optionally, adjusting the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera includes: when the load evaluation value is greater than a preset threshold, obtaining the reference load evaluation values of at least one adjacent camera adjacent to the target camera; adjusting the current region of interest of the target camera according to the current load evaluation value and the reference load evaluation values, and determining the target region of interest of the target camera.

[0055] When the current load evaluation value of the target camera is higher than a certain threshold and the current region of interest of the target camera needs to be adjusted, this application adjusts by referring to the load evaluation values of the cameras adjacent to the target camera, so as to avoid overloading adjacent cameras and reducing the performance and efficiency of adjacent cameras while reducing the current region of interest of the target camera.

[0056] Exemplarily, the current region of interest of the target camera can be gradually adjusted in a preset step according to the current load evaluation value and the reference load evaluation value, and the load evaluation values of the target camera and its adjacent cameras after adjustment are determined until an optimal solution is obtained in which both the target camera and its adjacent cameras can work normally.

[0057] An embodiment of the present invention provides a load balancing method for a multi-camera system. The method determines the current region of interest of a target camera according to the pre-calibrated overlapping region among the cameras in the multi-camera system; obtains the current defect rate detected by the target camera within a target time period; determines the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; and adjusts the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera. According to this technical solution, by monitoring the defect rate and load detected by each camera in real time, the region of interest of each camera is dynamically adjusted, so that the multi-camera system achieves load balancing, and the application performance and operation efficiency of the multi-camera system in industrial vision quality inspection are improved.

[0058] On the basis of the above embodiment, optionally, after obtaining the current defect rate detected by the target camera within a target time period, the method further includes: adjusting the current task amount of the target camera according to the current defect rate to determine the target task amount.

[0059] The tasks of the target camera may include various types such as image acquisition, image preprocessing, target detection and recognition, data transmission, and interaction with other devices or systems.

[0060] The current task amount of the target camera may be the task amount pre-allocated by the system. For example, the current task amount of the target camera includes image acquisition, image preprocessing, and target detection and recognition.

[0061] When the defect rate of the target camera is too high, that is, the load power required for the target camera to perform the target detection and recognition task is relatively high, the performance of the target camera will be reduced. Therefore, in this application, when the current defect rate of the target camera is higher than a certain threshold, part or all of the tasks responsible for the overlapping region between the target camera and the adjacent camera can be assigned to the adjacent camera to reduce the load of the target camera and improve the performance of the target camera.

[0062] Embodiment Two

[0063] Figure 2 It is a flowchart of a method for calibrating an overlapping region provided in Embodiment Two of this application. This embodiment is optimized based on the above embodiment. Specifically, it is the process of determining the overlapping region when the camera is a line-scan camera. As Figure 2 shown, the method of this embodiment specifically includes the following steps.

[0064] S210. Based on the black lines placed in advance in the overlapping region among the cameras, perform contour projection on the single-line pixels scanned by each camera at a target moment to generate a contour projection map; wherein, the contour projection map is a correspondence map between the pixel values and pixel coordinates in the single-line pixels.

[0065] Since the pixel value of the black line is very low and close to 0 in the camera imaging, in order to facilitate the observation and calibration of the overlapping area, in this application, a black thin linear object, such as a black narrow cable tie, a black narrow tape, etc., is manually placed in the overlapping area. It should be noted that in order to minimize the influence of the width of the black line itself on the overlapping area, a black line with a relatively narrow width should be selected.

[0066] In this application, each camera in the multi-camera system is defined as a line-scan camera. Its imaging principle is that when the object to be measured passes through the line-scan camera, the mechanical lens starts to move and transmits the image of the object to be measured to the photosensitive device at a set speed. The photosensitive units of the photosensitive device scan line by line, and convert the image information of each line into an electrical signal. The electrical signal is transmitted to an external image processor and forms a complete image after processing.

[0067] Therefore, in this application, the single-line pixels scanned by the target camera at a certain moment are acquired, and the overlapping area is calibrated in combination with the black line.

[0068] Specifically, in this application, the single-line pixels scanned by the camera are subjected to contour projection to generate a contour projection map. As Figure 3 shown, it is a schematic diagram of a contour projection provided by the second embodiment of this application. Its abscissa is the pixel coordinate, and the ordinate is the pixel value. The pixel coordinate is related to the resolution of the camera. The higher the resolution, the larger the pixel coordinate range; while the range of the pixel value is between [0, 255].

[0069] S220. For each of the cameras, according to the contour projection map, determine the pixel length of the visible area of the camera and the pixel coordinate corresponding to the minimum pixel value in the contour projection map.

[0070] It should be noted that since the black line is a manually placed black object, its pixel value is not necessarily 0 and may also be close to 0. Therefore, in this application, the pixel coordinate corresponding to the minimum pixel value in the contour projection map is used as the relative position of the black line within the visible range of the current camera.

[0071] S230. For the first camera and the second camera among adjacent cameras, according to the pixel length of the visible area of the first camera, the first pixel coordinate corresponding to the minimum pixel value of the first camera in the contour projection map, and the second pixel coordinate corresponding to the minimum pixel value of the second camera in the contour projection map, determine the overlapping area of the adjacent cameras. Among them, the first camera is located on the left side of the second camera.

[0072] For the contour projection map of each camera, its pixel coordinates are determined based on the visible range of the camera. Therefore, if the overlapping area is calibrated, the relative position of the black line in the visible area of the first camera and the relative position of the black line in the visible area of the second camera are considered together to determine the overlapping area of adjacent cameras.

[0073] Optionally, according to the pixel length of the visible area of the first camera in the contour projection map, the first pixel coordinate corresponding to the minimum pixel value of the first camera in the contour projection map, and the second pixel coordinate corresponding to the minimum pixel value of the second camera in the contour projection map, to determine the overlapping area of the adjacent cameras, including: determining a first distance from the black line to the right boundary of the visible area of the first camera according to the pixel length of the visible area of the first camera and the first pixel coordinate corresponding to the minimum pixel value of the first camera in the contour projection map; determining a second distance from the black line to the left boundary of the visible area of the second camera according to the second pixel coordinate corresponding to the minimum pixel value of the second camera in the contour projection map; and determining the overlapping area of the adjacent cameras according to the first distance and the second distance.

[0074] Figure 4 This is a working schematic diagram of adjacent cameras provided in the second embodiment of the present application. As Figure 4 shown, the visible range of the first camera Camera1 is , while the visible range of the second camera Camera2 is , and the overlapping area of the first camera and the second camera is .

[0075] At the target moment, perform contour projection on the single-row pixels scanned by the first camera to generate a contour projection map, as Figure 5 shown, Figure 5 which is the contour projection map of the first camera provided in the second embodiment of the present application; and perform contour projection on the single-row pixels scanned by the second camera to generate a contour projection map, as Figure 6 shown, Figure 6 which is the contour projection map of the second camera provided in the second embodiment of the present application.

[0076] According to Figure 5 , it can be known that the pixel length of the visible area of the first camera is 8200, and the coordinate value corresponding to the relative pixel coordinate point A1 of the black line in the visible area of the first camera is 7800; according to Figure 6 , it can be known that the coordinate value corresponding to the relative pixel coordinate point A2 of the black line in the visible area of the second camera is 800.

[0077] In this application, the pixel length of the visible area of the first camera can be subtracted from the first pixel coordinate corresponding to the minimum pixel value of the first camera in the contour projection diagram to obtain the first distance from the black line to the right boundary of the visible area of the first camera. In Figure 5 it, the first distance from the black line to the right boundary of the visible area of the first camera is 8200 - 7800 = 400.

[0078] Furthermore, the second pixel coordinate corresponding to the minimum pixel value of the second camera in the contour projection diagram is used as the second distance from the black line to the left boundary of the visible area of the second camera. In Figure 6 it, the second distance from the black line to the left boundary of the visible area of the second camera is 800.

[0079] Finally, the first distance and the second distance are added together to obtain the pixel length of the overlapping area between adjacent cameras. Furthermore, based on the pixel length of the visible area of the first camera and the pixel length of the overlapping area, the range of the overlapping area in the visible area of the first camera is obtained, and based on the pixel length of the visible area of the second camera and the pixel length of the overlapping area, the range of the overlapping area in the visible area of the second camera is obtained.

[0080] An embodiment of the present invention provides a method for calibrating an overlapping area. The method generates a contour projection diagram by performing contour projection on the single-row pixels scanned by each camera at a target moment based on a black line pre-placed in the overlapping area between each camera; wherein, the contour projection diagram is a correspondence diagram between the pixel value and the pixel coordinate in the single-row pixels; for each camera, according to the contour projection diagram, the pixel length of the visible area of the camera and the pixel coordinate corresponding to the minimum pixel value in the contour projection diagram are determined; for the first camera and the second camera among adjacent cameras, according to the pixel length of the visible area of the first camera, the first pixel coordinate corresponding to the minimum pixel value of the first camera in the contour projection diagram, and the second pixel coordinate corresponding to the minimum pixel value of the second camera in the contour projection diagram, the overlapping area between adjacent cameras is determined; wherein, the first camera is located on the left side of the second camera. This technical solution realizes the calibration of the overlapping area through the relative position of the black line in adjacent cameras, ensures the accuracy and consistency of image stitching, and thus improves the recognition accuracy of the multi-camera system.

[0081] Embodiment III

[0082] Figure 7The structural schematic diagram of a device 10 that can be used to implement the embodiments of the present application is shown. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0083] As Figure 7 shown, the device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0084] A plurality of components in the device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0085] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the load balancing method of the multi-camera system.

[0086] In some embodiments, the load balancing method of the multi-camera system can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the load balancing method of the multi-camera system described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the load balancing method of the multi-camera system by any other suitable means (e.g., by means of firmware).

[0087] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] The computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of this application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide for interaction with a user, the systems and techniques described herein can be implemented on a device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0091] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0092] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is generated by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0093] It should be understood that various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps recited in this application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of this application can be achieved, and no limitation is made herein.

[0094] The above specific embodiments do not constitute a limitation on the protection scope of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A load balancing method for a multi-camera system, characterized in that, The method includes: Determining a current region of interest of a target camera according to a pre-calibrated overlapping region among cameras in a multi-camera system; wherein, each camera in the multi-camera system is used to detect defects of a target product; Obtaining a current defect rate detected by the target camera within a target time period; Determining a current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; Adjusting the current region of interest of the target camera according to the current load evaluation value to determine a target region of interest of the target camera; Wherein, adjusting the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera includes: When the load evaluation value is greater than a preset threshold, obtaining a reference load evaluation value of at least one adjacent camera adjacent to the target camera; Gradually adjusting the current region of interest of the target camera according to the current load evaluation value and the reference load evaluation value by a preset step size until an optimal solution where both the target camera and the adjacent camera can be in a normal working state is obtained, and determining the target region of interest of the target camera.

2. The method according to claim 1, wherein Each of the cameras is a line scan camera; Correspondingly, the method for calibrating the overlapping region includes: Based on black lines pre-placed in the overlapping region among the cameras, respectively performing contour projection on the single-row pixels scanned by each camera at a target moment to generate a contour projection map; wherein, the contour projection map is a correspondence map between pixel values and pixel coordinates in the single-row pixels; For each camera, determining a pixel length of a visible region of the camera and pixel coordinates corresponding to a pixel minimum value in the contour projection map according to the contour projection map; For a first camera and a second camera among adjacent cameras, determining the overlapping region of the adjacent cameras according to the pixel length of the visible region of the first camera, a first pixel coordinate corresponding to the pixel minimum value of the first camera in the contour projection map, and a second pixel coordinate corresponding to the pixel minimum value of the second camera in the contour projection map; wherein, the first camera is located on the left side of the second camera.

3. The method according to claim 2, wherein Determining the overlapping region of the adjacent cameras according to the pixel length of the visible region of the first camera, a first pixel coordinate corresponding to the pixel minimum value of the first camera in the contour projection map, and a second pixel coordinate corresponding to the pixel minimum value of the second camera in the contour projection map includes: Determining a first distance from the black line to the right boundary of the visible region of the first camera according to the pixel length of the visible region of the first camera and the first pixel coordinate corresponding to the pixel minimum value of the first camera in the contour projection map; Determining a second distance from the black line to the left boundary of the visible region of the second camera according to the second pixel coordinate corresponding to the pixel minimum value of the second camera in the contour projection map; Determining the overlapping region of the adjacent cameras according to the first distance and the second distance.

4. The method according to claim 1, wherein Obtaining the current defect rate detected by the target camera within a target time period includes: Obtain the defective area and the total area detected by the target camera within a preset time period; Determine the current defect rate detected by the target camera within the target time period according to the defective area and the total area.

5. The method according to claim 1, wherein After obtaining the current defect rate detected by the target camera within the target time period, the method further includes: Adjust the current task volume of the target camera according to the current defect rate to determine the target task volume.

6. The method according to claim 1, wherein Before determining the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera, the method further includes: Obtain the historical defect rates of the target camera in a plurality of historical time periods, and determine the maximum defect rate and the minimum defect rate among the historical defect rates; Normalize the current defect rate according to the maximum defect rate and the minimum defect rate to determine the current standard defect rate.

7. An electronic device, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the load balancing method of the multi-camera system according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the load balancing method of the multi-camera system according to any one of claims 1-6 when executed by a processor.

9. A computer program product, characterized in that, The computer program product includes a computer program, and the computer program implements the load balancing method of the multi-camera system according to any one of claims 1-6 when executed by a processor.

Images