Load balancing method and device for multi-camera system, medium and program product

Abstract

The invention discloses a load balancing method and device for a multi-camera system, a medium and a program product. The method comprises the following steps: determining a current region of interest of a target camera according to an overlapping region calibrated among cameras in the multi-camera system in advance; obtaining a current defect rate detected by the target camera in the target time period; determining a current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; and adjusting the current region of interest of the target camera according to the current load evaluation value, and determining a target region of interest of the target camera. According to the technical scheme, the defect rate detected by each camera and the load are monitored in real time, so that the region of interest of each camera is dynamically adjusted, the load of the multi-camera system is balanced, and the application performance and the operation efficiency of the multi-camera system are improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a load balancing method, device, medium and program product for a multi-camera system. Background Art

[0002] Multi-camera systems have been widely used in many fields such as industrial production, intelligent transportation, medical image processing, and autonomous driving due to their wide coverage of viewing angles and high detection accuracy. When a multi-camera system is operating, in order to ensure that the multi-camera system can fully cover the detection surface, there needs to be a certain pixel overlap area between adjacent cameras in the multi-camera system to achieve seamless connection and data fusion of the detection surface.

[0003] At present, the processing methods for pixel overlap areas are mainly divided into two categories: feature point-based and frequency domain-based. The former is to extract image feature points and match corresponding point pairs to perform image stitching; the latter is to perform frequency domain analysis on the image to achieve a more natural stitching effect. Although the above methods have solved the problem of image stitching to a certain extent, when processing large-scale image stitching in complex environments, there are still problems such as unnatural stitching and low efficiency. In addition, the processing complexity of pixel overlap areas is relatively high, and often requires large computing resources, resulting in unbalanced load in multi-camera systems. If the load of a camera in a multi-camera system is too high, it may cause its detection delay or missed detection, thereby affecting the overall detection effect of the multi-camera system.

[0004] Therefore, how to provide a technical solution that can realize multi-camera intelligent collaborative operation is a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the invention

[0005] The present application provides a load balancing method, device, medium and program product for a multi-camera system. By real-time monitoring of the defect rate and load detected by each camera, dynamic adjustment of the area of ​​interest of each camera can be achieved, so that the multi-camera system can achieve load balancing and improve the application performance and operating efficiency of the multi-camera system in industrial visual quality inspection.

[0006] According to one aspect of the present application, a load balancing method for a multi-camera system is provided, the method comprising:

[0007] Determine the current region of interest of the target camera according to the overlap region pre-calibrated between the cameras in the multi-camera system; wherein each camera in the multi-camera system is used to detect defects of the target product;

[0008] Obtaining a current defect rate detected by the target camera within a target time period;

[0009] Determining a current load evaluation value of the target camera according to the current defect rate and the current load of the target camera;

[0010] According to the current load evaluation value, the current region of interest of the target camera is adjusted to determine the target region of interest of the target camera.

[0011] According to another aspect of the present application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the load balancing method for a multi-camera system described in any embodiment of the present application.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the load balancing method for a multi-camera system described in any embodiment of the present application when executed.

[0013] According to another aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, and the computer program, when executed by a processor, implements the load balancing method for a multi-camera system described in any embodiment of the present application.

[0014] The technical solution provided by the present application determines the current region of interest of the target camera based on the overlapped regions pre-calibrated between the cameras in the multi-camera system; obtains the current defect rate detected by the target camera in the target time period; determines the current load evaluation value of the target camera based on the current defect rate and the current load of the target camera; and adjusts the current region of interest of the target camera based on the current load evaluation value to determine the target region of interest of the target camera. This technical solution monitors the defect rate and load detected by each camera in real time to dynamically adjust the region of interest of each camera, so that the multi-camera system achieves load balancing and improves the application performance and operating efficiency of the multi-camera system in industrial visual quality inspection.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is a flow chart of a load balancing method for a multi-camera system provided in Example 1 of the present application.

[0018] Figure 2 A flowchart of a method for calibrating an overlap area provided in Example 2 of the present application.

[0019] Figure 3 A schematic diagram of a contour projection provided in Example 2 of the present application.

[0020] Figure 4 A schematic diagram of the operation of adjacent cameras provided in Embodiment 2 of the present application.

[0021] Figure 5 This is a contour projection diagram of the first camera provided in Example 2 of the present application.

[0022] Figure 6 This is a contour projection diagram of the second camera provided in Example 2 of the present application.

[0023] Figure 7 It is a structural diagram of a device for implementing a load balancing method for a multi-camera system in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "target", "first", "second", "current", "reference", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1 This is a flow chart of a load balancing method for a multi-camera system provided in the first embodiment of the present application. This embodiment is applicable to the operation of a multi-camera system. The method can be executed by a load balancing device for a multi-camera system. The load balancing device for a multi-camera system can be implemented in the form of hardware and / or software. The load balancing device for a multi-camera system can be configured in a device with data processing capabilities. Figure 1 As shown, the method includes the following steps.

[0028] S110, determining a current region of interest of a target camera according to a pre-calibrated overlap region between cameras in a multi-camera system, wherein each camera in the multi-camera system is used to detect defects of a target product.

[0029] A multi-camera system can be a system composed of multiple smart cameras. For example, in the field of industrial production, when manufacturing semiconductors, PCBs, circuit boards or other precision devices, a multi-camera system can be used to perform large-area and fine inspections of products on the production line. Another example is in the field of smart transportation, a multi-camera system can be used to achieve functions such as road condition monitoring, vehicle identification, and traffic violation detection.

[0030] In this application, a multi-camera system can be used to detect defects in a target product. For example, in a semiconductor production process, a camera captures images of wafers on a production line and identifies and detects defects in the wafer images, such as scratches, bubbles, cracks, and the like.

[0031] Since it is difficult for a single camera to cover a large detection area, in a multi-camera system, multiple cameras are combined and the pixel overlap area between the cameras is used to achieve full scene coverage and seamless detection. In this application, the overlap area between the cameras can be calibrated in advance according to the placement position of each camera and / or the actual visible area range.

[0032] The current region of interest can be a region of interest determined in the image currently captured by the target camera according to the actual application scenario and task requirements. For example, the overlap range of two adjacent cameras is (2300, 3700). When the initial conditions of the two cameras are the same, the region of interest of the left camera can be set to (2300, 3000) and the region of interest of the right camera can be set to (3000, 3700).

[0033] S120: Obtain a current defect rate detected by the target camera within a target time period.

[0034] The current defect rate can be the defect rate obtained by the target camera detecting the target product within the target time period. The higher the defect rate detected by the camera, the higher the power required by the camera when performing the defect recognition task; conversely, the lower the power required. This application makes it easier to balance the load of each camera in the multi-camera system by collecting statistics on the current defect rate.

[0035] For example, the defect rate can be expressed as the ratio of the number of defective products to the total number of products, or as the ratio of the number of defects, or as the ratio of the defective area to the total product area.

[0036] Specifically, the images captured by the target camera within the target time period can be obtained first, and then the number of defective products, defective areas or number of defects in the captured images can be counted, and finally the defect rate detected by the target camera within the target time period can be determined based on the statistical data.

[0037] Optionally, obtaining the current defect rate detected by the target camera within the target time period includes: obtaining the defect area and the total area detected by the target camera within a preset time period; and determining the current defect rate detected by the target camera within the target time period based on the defect area and the total area.

[0038] Specifically, the step length of this application is to count the defect area within the target time period in segments with the step length stride. The defect area detected in , in the target time period The total area detected within is , then the current defect rate It can be expressed by the following formula: .

[0039] In the present application, the detection data of each camera in the multi-camera system can be periodically acquired through sampling, the defect rate in each time period can be calculated and a corresponding histogram can be generated.

[0040] S130. Determine a current load evaluation value of the target camera according to the current defect rate and the current load of the target camera.

[0041] The current load may be the CPU usage rate, memory occupancy rate, or number of image frames processed per second of the target camera, etc. For example, the camera CPU usage rate is used to represent the current load of the target camera.

[0042] In order to accurately process the current defect rate and the current load, the current defect rate and the current load may be normalized respectively. For example, the current defect rate and the current load may be normalized by using a linear normalization method, a standard normalization method, a logarithmic function normalization method, an inverse tangent function normalization method, etc.

[0043] Optionally, before determining the current load evaluation value of the target camera based on the current defect rate and the current load of the target camera, the method also includes: obtaining the historical defect rate of the target camera in several historical time periods, and determining the maximum defect rate and the minimum defect rate among the historical defect rates; normalizing the current defect rate based on the maximum defect rate and the minimum defect rate to determine the current standard defect rate.

[0044] The present application normalizes the current defect rate to reduce the error impact on the defect rate caused by the difference in the total detection area in different time periods, thereby improving the accuracy of subsequent load balancing adjustments.

[0045] Specifically, this application uses a linear normalization method to control the current defect rate between 0 and 1. The current standard defect rate can be expressed by the following formula:

[0046] ;

[0047] In the formula, represents the current standard defect rate, D represents the current defect rate, represents the minimum defect rate in the historical defect rates, Indicates the maximum defect rate among the historical defect rates.

[0048] Further, the current load evaluation value of the target camera is determined according to the current defect rate and the current load. The load evaluation value is used to indicate the load level of the camera. The higher the load evaluation value, the greater the load of the camera, and the camera performance will be relatively reduced.

[0049] In the present application, the normalized current defect rate and current load may be weightedly summed to obtain the current load evaluation value of the target camera; the current defect rate and current load may also be multiplied to obtain the current load evaluation value of the target camera.

[0050] S140. According to the current load evaluation value, adjust the current region of interest of the target camera to determine the target region of interest of the target camera.

[0051] In the present application, if the current load evaluation value of the target camera is higher than a certain threshold, the current area of ​​interest of the target camera can be reduced, for example, all or part of the current area of ​​interest of the target camera can be allocated to an adjacent camera; if the current load evaluation value of the target camera is lower than a certain threshold, the current area of ​​interest of the target camera can be increased, for example, part of the area of ​​interest of an adjacent camera can be taken over; if the current load evaluation value of the target camera is within a certain range, the current area of ​​interest of the target camera may not be adjusted.

[0052] Specifically, it can be adjusted dynamically according to the current load evaluation value and the actual needs of the scene. For example, it can be adjusted through the following formula: ;

[0053] In the formula, represents the target region of interest, Indicates the current area of ​​interest. and To adjust the parameters, dynamic adjustments can be made according to the actual needs of the scene.

[0054] Optionally, according to the current load evaluation value, the current region of interest of the target camera is adjusted to determine the target region of interest of the target camera, including: when the load evaluation value is greater than a preset threshold, obtaining a reference load evaluation value of at least one adjacent camera adjacent to the target camera; according to the current load evaluation value and the reference load evaluation value, adjusting the current region of interest of the target camera to determine the target region of interest of the target camera.

[0055] If the current load evaluation value of the target camera is higher than a certain threshold and the current area of ​​interest of the target camera needs to be adjusted, the present application makes adjustments by referring to the load evaluation values ​​of cameras adjacent to the target camera to avoid excessively increasing the load of adjacent cameras while reducing the current area of ​​interest of the target camera, thereby reducing the performance and efficiency of the adjacent cameras.

[0056] Exemplarily, the current region of interest of the target camera can be gradually adjusted according to a preset step size based on the current load evaluation value and the reference load evaluation value, and the load evaluation values ​​of the target camera and its adjacent cameras after adjustment are determined until an optimal solution is obtained in which the target camera and its adjacent cameras are both in normal working condition.

[0057] The embodiment of the present invention provides a load balancing method for a multi-camera system, which determines the current region of interest of a target camera according to the overlapped regions pre-calibrated between the cameras in the multi-camera system; obtains the current defect rate detected by the target camera in the target time period; determines the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; and adjusts the current region of interest of the target camera according to the current load evaluation value to determine the target region of interest of the target camera. This technical solution monitors the defect rate and load detected by each camera in real time to dynamically adjust the region of interest of each camera, so that the multi-camera system achieves load balancing and improves the application performance and operating efficiency of the multi-camera system in industrial visual quality inspection.

[0058] Based on the above embodiment, optionally, after obtaining the current defect rate detected by the target camera within the target time period, the method further includes: adjusting the current task volume of the target camera according to the current defect rate to determine the target task volume.

[0059] The tasks of target cameras may include image acquisition, image preprocessing, target detection and recognition, data transmission, and interaction with other devices or systems.

[0060] The current task volume of the target camera may be a task volume pre-allocated by the system. For example, the current task volume of the target camera includes image acquisition, image preprocessing, and target detection and recognition.

[0061] When the target camera defect rate is too high, that is, the load power required by the target camera to perform target detection and recognition tasks is high, which will cause the performance of the target camera to decrease. Therefore, when the current defect rate of the target camera is higher than a certain threshold, the present application can allocate part or all of the tasks that the target camera and the adjacent camera are responsible for in their overlapping area to the adjacent camera, so as to reduce the target camera load and improve the target camera performance.

[0062] Embodiment 2

[0063] Figure 2 This is a flow chart of a method for calibrating an overlap area provided in the second embodiment of the present application. This embodiment is optimized based on the above embodiment, specifically, it is a process for determining the overlap area when the camera is a line scan camera. Figure 2 As shown, the method of this embodiment specifically includes the following steps.

[0064] S210. Based on the black lines pre-placed in the overlapping areas between the cameras, contour projection is performed on the single rows of pixels scanned by the cameras at the target time to generate a contour projection diagram; wherein the contour projection diagram is a correspondence diagram between pixel values ​​and pixel coordinates in the single row of pixels.

[0065] Since the pixel value of black lines in camera imaging is very low, close to 0, in order to facilitate observation and calibration of the overlap area, this application manually places black thin line objects in the overlap area, such as black narrow cable ties, black narrow tapes, etc. It should be noted that in order to minimize the influence of the width of the black lines themselves on the overlap area, narrower black lines should be selected.

[0066] In this application, each camera in a multi-camera system is defined as a line scan camera, and its imaging principle is that when the object to be measured passes through the line scan camera, the mechanical lens starts to move and transmits the image of the object to be measured to the photosensitive device at a set speed. The photosensitive unit of the photosensitive device scans line by line and converts the image information of each line into an electrical signal, which is transmitted to an external image processor and forms a complete image after processing.

[0067] Therefore, the present application obtains a single row of pixels scanned by the target camera at a certain moment and calibrates the overlapping area in combination with the black lines.

[0068] Specifically, the present application performs contour projection on a single row of pixels scanned by the camera to generate a contour projection image. Figure 3 As shown, it is a schematic diagram of a contour projection provided by the second embodiment of the present application, wherein the horizontal coordinate is the pixel coordinate and the vertical coordinate is the pixel value. The pixel coordinate is related to the resolution of the camera. The higher the resolution, the larger the pixel coordinate range; and the pixel value range is between [0,255].

[0069] S220 . For each camera, determine the pixel length of the visible area of ​​the camera and the pixel coordinates corresponding to the pixel minimum value in the contour projection image according to the contour projection image.

[0070] It should be noted that since the black line is a black object placed artificially, its pixel value is not necessarily 0, but may be close to 0. Therefore, this application uses the pixel coordinates corresponding to the minimum pixel value in the contour projection image as the relative position of the black line within the current camera visual range.

[0071] S230: For a first camera and a second camera among adjacent cameras, determine an overlap area of ​​the adjacent cameras according to a pixel length of a visible area of ​​the first camera, a first pixel coordinate corresponding to a pixel minimum value of the first camera in a contour projection diagram, and a second pixel coordinate corresponding to a pixel minimum value of the second camera in a contour projection diagram, wherein the first camera is located on the left side of the second camera.

[0072] For each camera's outline projection, its pixel coordinates are determined based on the camera's visible range. Therefore, if the overlap area is calibrated, the overlap area of ​​adjacent cameras is determined by combining the relative position of the black line in the visible area of ​​the first camera and the relative position of the black line in the visible area of ​​the second camera.

[0073] Optionally, the overlapping area of ​​the adjacent cameras is determined according to the pixel length of the visible area of ​​the first camera in the contour projection image, the first pixel coordinate corresponding to the pixel minimum of the first camera in the contour projection image, and the second pixel coordinate corresponding to the pixel minimum of the second camera in the contour projection image, including: determining a first distance from the black line to the right edge of the visible area of ​​the first camera according to the pixel length of the visible area of ​​the first camera and the first pixel coordinate corresponding to the pixel minimum of the first camera in the contour projection image; determining a second distance from the black line to the left edge of the visible area of ​​the second camera according to the second pixel coordinate corresponding to the pixel minimum of the second camera in the contour projection image; and determining the overlapping area of ​​the adjacent cameras according to the first distance and the second distance.

[0074] Figure 4 This is a schematic diagram of the operation of adjacent cameras provided in the second embodiment of the present application. Figure 4 As shown, the visual range of the first camera Camera1 is , and the visible range of the second camera Camera2 is , the overlap area of ​​the first camera and the second camera is .

[0075] At the target time, a single row of pixels scanned by the first camera is contour projected to generate a contour projection map, such as Figure 5 As shown, Figure 5 The contour projection diagram of the first camera provided in the second embodiment of the present application; and contour projection is performed on a single row of pixels scanned by the second camera to generate a contour projection diagram, such as Figure 6 As shown, Figure 6 This is a contour projection diagram of the second camera provided in Example 2 of the present application.

[0076] according to Figure 5 It can be seen that the pixel length of the visible area of ​​the first camera is 8200, and the coordinate value corresponding to the relative pixel coordinate point A1 of the black line in the visible area of ​​the first camera is 7800; according to Figure 6 It can be seen that the coordinate value corresponding to the relative pixel coordinate point A2 of the black line in the visible area of ​​the second camera is 800.

[0077] In the present application, the first distance from the black line to the right edge of the visible area of ​​the first camera can be obtained by subtracting the first pixel coordinate corresponding to the pixel minimum of the first camera in the contour projection image from the pixel length of the visible area of ​​the first camera. Figure 5 In the figure, the first distance from the black line to the right edge of the visible area of ​​the first camera is 8200-7800=400.

[0078] Furthermore, the second pixel coordinate corresponding to the pixel minimum value of the second camera in the contour projection image is used as the second distance from the black line to the left edge of the visible area of ​​the second camera. Figure 6 , the second distance from the black line to the left border of the second camera's visible area is 800.

[0079] Finally, the first distance and the second distance are added to obtain the pixel length of the overlapping area of ​​adjacent cameras, and then the range of the overlapping area in the visible area of ​​the first camera is obtained according to the pixel length of the visible area of ​​the first camera and the pixel length of the overlapping area, and the range of the overlapping area in the visible area of ​​the second camera is obtained according to the pixel length of the visible area of ​​the second camera and the pixel length of the overlapping area.

[0080] The embodiment of the present invention provides a calibration method for overlapped areas. The method generates an outline projection diagram by performing outline projection on a single row of pixels scanned by each camera at a target time based on black lines pre-placed in the overlapped areas between the cameras; wherein the outline projection diagram is a corresponding relationship diagram between pixel values ​​and pixel coordinates in a single row of pixels; for each camera, the pixel length of the visible area of ​​the camera and the pixel coordinates corresponding to the pixel minimum in the outline projection diagram are determined according to the outline projection diagram; for the first camera and the second camera in the adjacent cameras, the overlapped area of ​​the adjacent cameras is determined according to the pixel length of the visible area of ​​the first camera, the first pixel coordinates corresponding to the pixel minimum in the outline projection diagram of the first camera, and the second pixel coordinates corresponding to the pixel minimum in the outline projection diagram of the second camera; wherein the first camera is located on the left side of the second camera. The technical solution realizes the calibration of the overlapped area through the relative position of the black lines in the adjacent cameras, ensures the accuracy and consistency of image stitching, and thus improves the recognition accuracy of the multi-camera system.

[0081] Embodiment 3

[0082] Figure 7A schematic diagram of a device 10 that can be used to implement an embodiment of the present application is shown. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0083] like Figure 7 As shown, the device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0084] A number of components in the device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0085] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a load balancing method for a multi-camera system.

[0086] In some embodiments, the load balancing method for a multi-camera system may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the load balancing method for a multi-camera system described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the load balancing method for a multi-camera system in any other appropriate manner (e.g., by means of firmware).

[0087] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flow charts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0089] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein may be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0092] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0093] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.

[0094] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A load balancing method for a multi-camera system, characterized in that: The method comprises: Determine the current region of interest of the target camera according to the overlap region pre-calibrated between the cameras in the multi-camera system; wherein each camera in the multi-camera system is used to detect defects of the target product; Obtaining a current defect rate detected by the target camera within a target time period; Determining a current load evaluation value of the target camera according to the current defect rate and the current load of the target camera; According to the current load evaluation value, the current region of interest of the target camera is adjusted to determine the target region of interest of the target camera.

2. The method according to claim 1, characterized in that Each camera is a line scan camera; Accordingly, the calibration method of the overlap area includes: Based on the black lines pre-placed in the overlapping areas between the cameras, contour projection is performed on the single-row pixels scanned by the cameras at the target time to generate a contour projection diagram; wherein the contour projection diagram is a corresponding relationship diagram between pixel values ​​and pixel coordinates in the single-row pixels; For each of the cameras, determining the pixel length of the visible area of ​​the camera and the pixel coordinates corresponding to the pixel minimum value in the contour projection map according to the contour projection map; For a first camera and a second camera among adjacent cameras, an overlapping area of ​​the adjacent cameras is determined according to a pixel length of a visible area of ​​the first camera, a first pixel coordinate corresponding to a pixel minimum value of the first camera in a contour projection image, and a second pixel coordinate corresponding to a pixel minimum value of the second camera in the contour projection image; wherein the first camera is located on the left side of the second camera.

3. The method according to claim 2, characterized in that Determining the overlap area of ​​the adjacent cameras according to the pixel length of the visible area of ​​the first camera, the first pixel coordinate corresponding to the pixel minimum value of the first camera in the contour projection image, and the second pixel coordinate corresponding to the pixel minimum value of the second camera in the contour projection image, includes: Determine a first distance from the black line to the right edge of the visible area of ​​the first camera according to the pixel length of the visible area of ​​the first camera and the first pixel coordinate corresponding to the pixel minimum value of the first camera in the contour projection image; Determine a second distance from the black line to the left edge of the visible area of ​​the second camera according to a second pixel coordinate corresponding to a pixel minimum value in the contour projection image of the second camera; An overlapping area of ​​the adjacent cameras is determined according to the first distance and the second distance.

4. The method according to claim 1, characterized in that: Obtaining the current defect rate detected by the target camera within a target time period, including: Obtaining the defect area and total area detected by the target camera within a preset time period; A current defect rate detected by the target camera within a target time period is determined according to the defect area and the total area.

5. The method according to claim 1, characterized in that After obtaining the current defect rate detected by the target camera within the target time period, the method further includes: According to the current defect rate, the current task volume of the target camera is adjusted to determine the target task volume.

6. The method according to claim 1, characterized in that Before determining the current load evaluation value of the target camera according to the current defect rate and the current load of the target camera, the method further includes: Obtaining historical defect rates of the target camera in several historical time periods, and determining a maximum defect rate and a minimum defect rate among the historical defect rates; The current defect rate is normalized according to the maximum defect rate and the minimum defect rate to determine a current standard defect rate.

7. The method according to claim 1, characterized in that According to the current load evaluation value, adjusting the current region of interest of the target camera to determine the target region of interest of the target camera includes: When the load evaluation value is greater than a preset threshold, obtaining a reference load evaluation value of at least one adjacent camera adjacent to the target camera; According to the current load evaluation value and the reference load evaluation value, the current region of interest of the target camera is adjusted to determine the target region of interest of the target camera.

8. An electronic device, characterized in that: The device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the load balancing method for a multi-camera system according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the load balancing method for a multi-camera system according to any one of claims 1 to 7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the load balancing method for a multi-camera system according to any one of claims 1 to 7.

Images