Automatic calibration method, device, equipment and storage medium based on multiple cameras

Abstract

The embodiments of the present invention relate to the technical field of position calibration, and disclose an automatic calibration method, device, equipment and storage medium based on multiple cameras. The method comprises: step 1, setting N cameras in the work site area; step 2, setting M calibration objects in multiple working spaces in the work site area, and recording the spatial coordinates of the calibration objects; step 3, using N cameras to shoot the M calibration objects to obtain the image coordinates of the calibration objects in the cameras; step 4, using the image coordinates of the calibration objects shot by the cameras as input, and the corresponding spatial coordinates of the calibration objects as target output, to train a pre-constructed neural network initial model to obtain the final neural network classifier; step 5, using the neural network classifier to identify the spatial position of the object to be located. By implementing the embodiments of the present invention, automatic calibration can be performed after the number and / or position of cameras are adjusted, without the need for manual recalibration.

Description

Technical Field

[0001] The present invention relates to the technical field of position calibration, and in particular to an automatic calibration method, device, equipment and storage medium based on multiple cameras. Background Art

[0002] Since the emergence of positioning technology in the last century, after years of development, a variety of methods have emerged. The most commonly used positioning method is satellite positioning technology, which uses artificial satellites to measure points. However, due to the limitations of positioning accuracy, this technology cannot accurately locate small objects. In addition to satellite positioning methods, the commonly used positioning methods include UWB (Ultra-Wide Band) positioning methods. This method can achieve accurate positioning of objects by arranging base stations at the positioning site, but this method requires a lot of effort in the early deployment of base stations. With the improvement of computer performance and the development of camera technology, it has become a reality to use cameras to accurately locate targets.

[0003] Using cameras is a common method for monitoring, and multi-camera positioning is also a common method for positioning. However, when using a multi-camera system for position calibration, the position of the camera in the working state cannot be changed, and the number of cameras must not be increased or decreased. Otherwise, the camera needs to be recalibrated, which will cause various problems in actual operation. Summary of the invention

[0004] In response to the above-mentioned defects, an embodiment of the present invention discloses an automatic calibration method, device, equipment and storage medium based on multiple cameras, which can automatically complete the calibration when the position and / or number of cameras are adjusted.

[0005] A first aspect of an embodiment of the present invention discloses an automatic calibration method based on multiple cameras, the method comprising:

[0006] Step 1, determining the work site area to be positioned, and setting N cameras in the work site area;

[0007] Step 2: Set M calibration objects in multiple working spaces in the work site area, and record the spatial coordinates (x i ,y i ), where 1≤i≤M;

[0008] Step 3: Use the N cameras to shoot the M calibration objects, and obtain the image coordinates (p ij ,q ij ), where 1≤j≤N;

[0009] Step 4: Take the image coordinates of the calibration object captured by the camera as input and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier;

[0010] Step 5, using the neural network classifier to identify the spatial position of the object to be located, includes:

[0011] When the number and position of cameras do not change, the image coordinates of the object to be located in each camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located;

[0012] When the number and / or position of cameras changes, the unchanged camera is defined as the first camera, the camera that is added and / or has its position changed is defined as the second camera, the image coordinates of the object to be located in the first camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located, and the neural network classifier is retrained according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera.

[0013] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the image coordinates of the calibration object captured by the camera are used as input, and the corresponding spatial coordinates of the calibration object are used as target output to train the pre-built neural network initial model to obtain the final neural network classifier, including:

[0014] For the total M×N groups of image coordinates determined in step 3, P groups of shielding vectors are randomly generated, wherein the shielding vectors contain a total of N elements, each element corresponds to a camera, and the value of the element is 1 or 0. When the value of the element is 1, it means that the image coordinates of the camera corresponding to the element remain unchanged; when the value of the element is 0, the image coordinates of the corresponding camera are set to the origin;

[0015] The image coordinates of the calibration object captured by the camera corresponding to the shielding vector are used as input information, and the corresponding spatial coordinates of the calibration object are used as target output to train the pre-built neural network initial model to obtain the final neural network classifier.

[0016] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the retraining of the neural network classifier according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera includes:

[0017] The element corresponding to the second camera in the shielding vector is set to 0, and a training data quantity threshold α is set;

[0018] The image coordinates of the object to be located in the first camera and the second camera are saved as new training data, and when the amount of new training data reaches a training data amount threshold α, the new training data is used as input information of the neural network classifier, and the spatial coordinates of the object to be located are used as target output to retrain the neural network classifier;

[0019] After the training is completed, the element corresponding to the second camera in the shielding vector is modified to 1.

[0020] As a preferred embodiment, in the first aspect of the embodiment of the present invention, when the second camera is a newly added camera, an element corresponding to the newly added camera is added to the shielding vector, and the element corresponding to the newly added camera in the shielding vector is set to 0.

[0021] As a preferred embodiment, in the first aspect of the embodiment of the present invention, when the number of cameras is reduced, the elements corresponding to the reduced cameras in the shielding vector are set to 0, and the image coordinates of the object to be located in other cameras are input into the neural network classifier to obtain the spatial coordinates of the object to be located.

[0022] A second aspect of an embodiment of the present invention discloses an automatic calibration device based on multiple cameras, which includes:

[0023] A setting unit, used for determining a work site area that needs to be positioned, and setting N cameras in the work site area;

[0024] The recording unit is used to set M calibration objects in multiple working spaces in the work site area and record the spatial coordinates (x i ,y i ), where 1≤i≤M;

[0025] A determination unit is configured to use the N cameras to shoot the M calibration objects, and obtain the image coordinates (p ij ,q ij ), where 1≤j≤N;

[0026] A training unit is used to take the image coordinates of the calibration object captured by the camera as input and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier;

[0027] A calibration unit, used to identify the spatial position of the object to be located using the neural network classifier, comprising:

[0028] When the number and position of cameras do not change, the image coordinates of the object to be located in each camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located;

[0029] When the number and / or position of cameras changes, the unchanged camera is defined as the first camera, and the camera that is added and / or has its position changed is defined as the second camera. The image coordinates of the object to be located in the first camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located. The neural network classifier is retrained according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera. The number of the first camera and the second camera can be one or more depending on the specific situation, and is not limited here.

[0030] As a preferred embodiment, in the second aspect of the embodiment of the present invention, the training unit includes:

[0031] A generating subunit, for randomly generating P groups of shielding vectors for a total of M×N groups of image coordinates determined in the determining unit, wherein the shielding vectors contain a total of N elements, each element corresponds to a camera, and the value of the element is 1 or 0. When the value of the element is 1, it means that the image coordinates of the camera corresponding to the element remain unchanged; when the value of the element is 0, the image coordinates of the corresponding camera are set to the origin;

[0032] The training subunit is used to take the image coordinates of the calibration object captured by the camera corresponding to the shielding vector as input information and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier.

[0033] As a preferred embodiment, in the second aspect of the embodiment of the present invention, the calibration unit includes:

[0034] A setting subunit, used to set the element corresponding to the second camera in the shielding vector to 0, and set a training data quantity threshold α;

[0035] a retraining subunit, configured to save the image coordinates of the object to be located in the first camera and the second camera as new training data, and when the amount of the new training data reaches a training data amount threshold α, use the new training data as input information of the neural network classifier and use the spatial coordinates of the object to be located as target output to retrain the neural network classifier;

[0036] The modification subunit is used to modify the element corresponding to the second camera in the shielding vector to 1 after the training is completed.

[0037] A third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute an automatic calibration method based on multiple cameras disclosed in the first aspect of the embodiment of the present invention.

[0038] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute an automatic calibration method based on multiple cameras disclosed in the first aspect of an embodiment of the present invention.

[0039] A fifth aspect of an embodiment of the present invention discloses a computer program product. When the computer program product runs on a computer, the computer executes an automatic calibration method based on multiple cameras disclosed in the first aspect of the embodiment of the present invention.

[0040] A sixth aspect of an embodiment of the present invention discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes an automatic calibration method based on multiple cameras disclosed in the first aspect of an embodiment of the present invention.

[0041] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0042] The embodiment of the present invention can complete automatic calibration after the number and / or position of cameras are adjusted by automatic training, without the need for manual recalibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technical personnel in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 It is a flowchart of an automatic calibration method based on multiple cameras disclosed in an embodiment of the present invention;

[0045] Figure 2 It is a structural diagram of the initial model of the neural network disclosed in the embodiment of the present invention.

[0046] Figure 3 It is a structural schematic diagram of an automatic calibration device based on multiple cameras disclosed in an embodiment of the present invention;

[0047] Figure 4 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in the field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] The embodiments of the present invention disclose a multi-camera based automatic calibration method, device, equipment and storage medium, which can automatically complete the calibration, and are described in detail below with reference to the accompanying drawings.

[0051] Embodiment 1

[0052] An embodiment of the present invention provides an automatic calibration solution for adding or removing cameras or changing their positions in a camera positioning system. By utilizing the redundancy of a multi-camera system, the system can be automatically calibrated after the cameras are added or removed or their positions are changed, without the need for manual intervention, making it more convenient to use.

[0053] The specific flow chart is as follows Figure 1 shown.

[0054] Step 1: Determine the work site area that needs to be positioned, and set up N cameras in the work site area.

[0055] The N cameras are preferably evenly distributed in the work site area, and the images they capture can cover the entire work site area. Wide-angle cameras can also be used, and the number of camera layouts should be no less than 3.

[0056] Step 2: Set up M calibration objects in multiple working spaces in the work site area, and record the spatial coordinates (x i ,y i )where 1≤i≤M.

[0057] It is preferred to set up at least one calibration object in each working space. The spatial coordinates can be the point coordinates in a coordinate system with the center of the work site area as the origin. Of course, they can also be the point coordinates in a coordinate system with any point in the work site area as the origin. They can also be the coordinates of the world coordinate system, which is not limited here.

[0058] Step 3: Use N cameras arranged in the work site area to shoot the M calibration objects set in the work space, and record the spatial coordinates of each calibration object in each camera image, and obtain the image coordinates of the i-th calibration object in the j-th camera (p ij ,q ij ), where 1≤i≤M, 1≤j≤N.

[0059] The calibration object is preferably an object or label with a large color distinction from the work site area. The image coordinates of each calibration object in the image taken by each camera can be determined by color recognition. The image coordinates can be pixel coordinates or point coordinates in a coordinate system with the center of the image as the origin, which is not limited here.

[0060] When determining the calibration object and the camera, it is preferred that at least one camera can capture one of the calibration objects, and the number of image coordinates obtained is M×N groups. Of course, when a camera does not capture the target calibration object, the image coordinates of the target calibration object in the camera are defined as (0, 0). In training the initial model of the neural network, the image coordinates are not used to train the initial model of the neural network.

[0061] In order to realize automatic calibration, a shielding vector is introduced in the embodiment of the present invention, please refer to step 4.

[0062] Step 4: For the total M×N groups of image coordinates determined in step 3 above, randomly generate P groups (P can be 1 or a positive integer greater than 1) of shielding vectors, where the shielding vector is in the form of [1 0 ··· 0 1], where the shielding vector contains N elements, each element corresponds to a camera, and the value of the element can be 1 or 0. When the value of an element in the shielding vector is 1, the image coordinates of the camera corresponding to the element remain unchanged; when the value of an element in the shielding vector is 0, the image coordinates of the corresponding camera are set to the origin (0,0). It is required that the number of elements with a value of 1 in the shielding vector is greater than the minimum number of cameras required for positioning (the minimum number of cameras is 3).

[0063] Step 5: Use the image coordinates of the calibration object captured by the camera corresponding to the above shielding vector as input, and the corresponding calibration object space coordinates as the target output to train the neural network initial model. The neural network model has strong nonlinear fitting ability, can map any complex nonlinear relationship, and has simple learning rules, which is easy to implement on a computer. It has strong robustness, memory ability, nonlinear mapping ability and strong self-learning ability. The data obtained from steps 3 and 4 above are used to train the neural network initial model. The structure of the neural network initial model is as follows: Figure 2 As shown, the specific steps for training the initial neural network model using the data obtained in steps 3 and 4 above are as follows:

[0064] Step 5.1: The entire neural network initial model consists of three parts, an input layer, a feature layer and an output layer, wherein the input layer is located in the first layer and the output layer is located in the last layer. For the present invention, the input information is the coordinate information of the calibration object in the image captured by the camera obtained in step 3, and the output information is the actual coordinates of the calibration object in the positioning space.

[0065] Step 5.2: After determining the input and output information, set the loss function and loss threshold of the initial model of the neural network. The loss function calculates the difference between the forward calculation result of each iteration of the neural network and the true value, so as to know that the next step of training is going in the right direction. After confirmation, start training the initial model of the neural network.

[0066] Step 5.3: In order to achieve better training results and make the neural network more robust, the structure of the intermediate feature layer of the neural network is appropriately adjusted until the loss function value of the neural network is less than the predetermined loss threshold. After the training is completed, a neural network classifier is obtained to identify the spatial coordinates of other objects to be located.

[0067] Step 6: When the entire camera system is in normal use (the number and / or position of cameras have not changed), all elements in the shielding matrix are set to 1, and the image coordinates of the object to be located in each camera (the position of the object to be located in the image taken by the camera can be determined by an artificial intelligence algorithm, and then the image coordinates are determined) are input into the above-mentioned neural network classifier to obtain the actual spatial coordinates of the object to be located. When adding or reducing cameras in the entire system, the camera positioning system proposed in the embodiment of the present invention will perform the following operations to complete automatic calibration.

[0068] Step 6.1: When a camera is reduced in the system, the element value corresponding to the camera in the shielding vector is set to 0, and the image coordinates of the object to be located determined by other cameras are input into the neural network classifier to obtain the spatial coordinate output of the object to be located. It can be understood that when the number of cameras in the system is reduced to 2 or less, the spatial coordinate output of the object to be located is no longer performed, and a reminder message can be issued by alarm.

[0069] Step 6.2: When a camera is added to the system, increase the number of dimensions of the shielding vector, and set one dimension to correspond to the added camera, and set the element in the shielding matrix to 0. Use the original camera to detect the object to be located and determine the spatial coordinates of the object, and save the image coordinates of the object to be located in the original camera and the newly added camera as new training data. Set the training data quantity threshold α. When the newly collected training data reaches the preset quantity, repeat the above steps 4 and 5 to retrain the new neural network classifier. After the training is completed, modify the element in the shielding matrix corresponding to the added camera to 1.

[0070] Of course, it is understandable that if the set threshold α is greater than the number of image coordinates of the objects to be located in the original camera and the newly added camera, the image coordinate information of the calibration object can be used as part of the new training data. If there is no object to be located, any one or more calibration objects can be used as the object to be located to automatically retrain the neural network classifier.

[0071] In other words, when the number and / or positions of cameras are adjusted, the spatial coordinates of the objects to be located can still be automatically identified. In order to ensure accuracy, except for the case where the number of cameras is reduced, the neural network classifier can be automatically retrained according to the objects to be located and / or the calibration objects.

[0072] Step 6.3: When the position of a camera in the system changes, set the element of the camera whose GIA position changes in the shielding vector to 0, use other cameras to detect and locate the object and determine the spatial coordinates of the object, and save the image coordinates of the object to be located in other cameras and the newly added camera as new training data. Set the training data quantity threshold α, when the newly collected training data reaches the preset quantity, repeat the above steps 4 and 5 to retrain the new neural network classifier, and after the training is completed, modify the element in the shielding matrix corresponding to the camera whose position changes to 1.

[0073] It is understandable that when a combination of the above three situations occurs, the implementation principle is similar to that of each implementation. For example, when a camera (called the second camera) is damaged and a camera (called the third camera) changes position in the system, the elements in the shielding matrix corresponding to the second camera and the third camera are modified to 0, and then the other camera (called the first camera) is used to detect and locate the object and determine the spatial coordinates of the object, and the image coordinates of the object to be located in the first camera and the third camera are saved as new training data. Set the training data quantity threshold α, when the newly collected training data reaches the preset quantity, repeat the above steps 4 and 5 to retrain the new neural network classifier, after the training is completed, modify the elements in the shielding matrix corresponding to the third camera to 1, and the elements in the shielding matrix corresponding to the second camera are still 0. Of course, the elements in the shielding matrix corresponding to the second camera can also be directly removed from the shielding matrix.

[0074] In summary, by implementing the embodiments of the present invention, the system can be automatically calibrated after the number of cameras is increased, reduced, or the position is changed, without the need for manual intervention, and is more convenient to use.

[0075] Embodiment 2

[0076] See also Figure 3 , Figure 3 Schematic diagram of a multi-camera based automatic calibration device disclosed in an embodiment of the present invention. Figure 3 As shown, the automatic calibration device based on multiple cameras may include:

[0077] A setting unit 210 is used to determine a work site area that needs to be positioned, and to set N cameras in the work site area;

[0078] The recording unit 220 is used to set M calibration objects in multiple working spaces in the work site area, and record the spatial coordinates (x i ,y i ), where 1≤i≤M;

[0079] The determination unit 230 is configured to use the N cameras to shoot the M calibration objects, and obtain the image coordinates (p ij ,q ij ), where 1≤j≤N;

[0080] The training unit 240 is used to take the image coordinates of the calibration object captured by the camera as input and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier;

[0081] The calibration unit 250 is used to identify the spatial position of the object to be located by using the neural network classifier, and includes:

[0082] When the number and position of cameras do not change, the image coordinates of the object to be located in each camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located;

[0083] When the number and / or position of cameras changes, the unchanged camera is defined as the first camera, the camera that is added and / or has its position changed is defined as the second camera, the image coordinates of the object to be located in the first camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located, and the neural network classifier is retrained according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera.

[0084] Preferably, the training unit may include:

[0085] A generating subunit, for randomly generating P groups of shielding vectors for a total of M×N groups of image coordinates determined in the determining unit, wherein the shielding vectors contain a total of N elements, each element corresponds to a camera, and the value of the element is 1 or 0. When the value of the element is 1, it means that the image coordinates of the camera corresponding to the element remain unchanged; when the value of the element is 0, the image coordinates of the corresponding camera are set to the origin;

[0086] The training subunit is used to take the image coordinates of the calibration object captured by the camera corresponding to the shielding vector as input information and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier.

[0087] Preferably, the calibration unit may include:

[0088] A setting subunit, used to set the element corresponding to the second camera in the shielding vector to 0, and set a training data quantity threshold α;

[0089] a retraining subunit, configured to save the image coordinates of the object to be located in the first camera and the second camera as new training data, and when the amount of the new training data reaches a training data amount threshold α, use the new training data as input information of the neural network classifier and use the spatial coordinates of the object to be located as target output to retrain the neural network classifier;

[0090] The modification subunit is used to modify the element corresponding to the second camera in the shielding vector to 1 after the training is completed.

[0091] Embodiment 3

[0092] See also Figure 4 , Figure 4 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Figure 4 As shown, the electronic device may include:

[0093] A memory 310 storing executable program codes;

[0094] a processor 320 coupled to the memory 310;

[0095] The processor 320 calls the executable program code stored in the memory 310 to execute part or all of the steps in the automatic calibration method based on multiple cameras in the first embodiment.

[0096] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in an automatic calibration method based on multiple cameras in Embodiment 1.

[0097] An embodiment of the present invention further discloses a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in an automatic calibration method based on multiple cameras in embodiment one.

[0098] An embodiment of the present invention also discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in an automatic calibration method based on multiple cameras in embodiment one.

[0099] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0103] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0104] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0105] The above is a detailed introduction to the automatic calibration method, device, equipment and storage medium based on multiple cameras disclosed in the embodiments of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An automatic calibration method based on multiple cameras, characterized in that: include: Step 1, determining the work site area to be positioned, and setting N cameras in the work site area; Step 2: Set M calibration objects in multiple working spaces in the work site area, and record the spatial coordinates (x i ,y i ), where 1≤i≤M; Step 3: Use the N cameras to shoot the M calibration objects, and obtain the image coordinates (p ij ,q ij ), where 1≤j≤N; Step 4: Take the image coordinates of the calibration object captured by the camera as input and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier; Step 5, using the neural network classifier to identify the spatial position of the object to be located, which includes: When the number and position of cameras do not change, the image coordinates of the object to be located in each camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located; When the number and / or position of the cameras change, the unchanged camera is defined as the first camera, the camera that is added and / or whose position changes is defined as the second camera, the image coordinates of the object to be located in the first camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located, and the neural network classifier is retrained according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera; The image coordinates of the calibration object captured by the camera are used as input, and the corresponding spatial coordinates of the calibration object are used as target output to train the pre-built neural network initial model to obtain the final neural network classifier, including: For the total M×N groups of image coordinates determined in step 3, P groups of shielding vectors are randomly generated, wherein the shielding vectors contain a total of N elements, each element corresponds to a camera, and the value of the element is 1 or 0. When the value of the element is 1, it means that the image coordinates of the camera corresponding to the element remain unchanged; when the value of the element is 0, the image coordinates of the corresponding camera are set to the origin; The image coordinates of the calibration object captured by the camera corresponding to the shielding vector are used as input information, and the corresponding spatial coordinates of the calibration object are used as target output to train the pre-built neural network initial model to obtain the final neural network classifier.

2. The multi-camera based automatic calibration method according to claim 1, characterized in that: The retraining of the neural network classifier according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera comprises: The element corresponding to the second camera in the shielding vector is set to 0, and a training data quantity threshold α is set; The image coordinates of the object to be located in the first camera and the second camera are saved as new training data, and when the amount of the new training data reaches a training data amount threshold a, the new training data is used as input information of the neural network classifier, and the spatial coordinates of the object to be located are used as target output to retrain the neural network classifier; After the training is completed, the element corresponding to the second camera in the shielding vector is modified to 1.

3. The multi-camera based automatic calibration method according to claim 2, characterized in that: When the second camera is a newly added camera, an element corresponding to the newly added camera is added to the shielding vector, and the element corresponding to the newly added camera in the shielding vector is set to 0.

4. The multi-camera based automatic calibration method according to claim 1, characterized in that: When the number of cameras decreases, the elements corresponding to the reduced cameras in the shielding vector are set to 0, and the image coordinates of the object to be located in other cameras are input into the neural network classifier to obtain the spatial coordinates of the object to be located.

5. An automatic calibration device based on multiple cameras, characterized in that: It includes: A setting unit, used for determining a work site area that needs to be positioned, and setting N cameras in the work site area; The recording unit is used to set M calibration objects in multiple working spaces in the work site area and record the spatial coordinates (x i ,y i ), where 1≤i≤M; A determination unit is configured to use the N cameras to shoot the M calibration objects, and obtain the image coordinates (p ij ,q ij ), where 1≤j≤N; A training unit is used to take the image coordinates of the calibration object captured by the camera as input and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier; A calibration unit, used to identify the spatial position of the object to be located using the neural network classifier, comprising: When the number and position of cameras do not change, the image coordinates of the object to be located in each camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located; When the number and / or position of the cameras change, the unchanged camera is defined as the first camera, the camera that is added and / or whose position changes is defined as the second camera, the image coordinates of the object to be located in the first camera are input into the neural network classifier to obtain the spatial coordinates of the object to be located, and the neural network classifier is retrained according to the spatial coordinates of the object to be located and the image coordinates of the object to be located in the first camera and the second camera; The training unit comprises: A generating subunit, for randomly generating P groups of shielding vectors for a total of M×N groups of image coordinates determined in the determining unit, wherein the shielding vectors contain a total of N elements, each element corresponds to a camera, and the value of the element is 1 or 0. When the value of the element is 1, it means that the image coordinates of the camera corresponding to the element remain unchanged; when the value of the element is 0, the image coordinates of the corresponding camera are set to the origin; The training subunit is used to take the image coordinates of the calibration object captured by the camera corresponding to the shielding vector as input information and the corresponding spatial coordinates of the calibration object as target output to train the pre-built neural network initial model to obtain the final neural network classifier.

6. The multi-camera based automatic calibration device according to claim 5, characterized in that: The calibration unit comprises: A setting subunit, used to set the element corresponding to the second camera in the shielding vector to 0, and set a training data quantity threshold α; a retraining subunit, configured to save the image coordinates of the object to be located in the first camera and the second camera as new training data, and when the amount of the new training data reaches a training data amount threshold α, use the new training data as input information of the neural network classifier and use the spatial coordinates of the object to be located as target output to retrain the neural network classifier; The modification subunit is used to modify the element corresponding to the second camera in the shielding vector to 1 after the training is completed.

7. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the automatic calibration method based on multiple cameras as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the multi-camera based automatic calibration method as described in any one of claims 1 to 4.

Images