Camera detection method and device, electronic equipment and vehicle

Abstract

The present disclosure provides a camera detection method and device, electronic equipment and vehicle. The method comprises: acquiring images captured by each camera at the same time, each image captured by a camera partially overlaps with images captured by at least two other cameras; determining a target object according to the overlapping part of the images, the target object being the same object corresponding to at least two cameras; establishing a coordinate system; calculating first coordinate information of the target object in the coordinate system according to the corresponding at least two cameras, or calculating second coordinate information of the corresponding at least two cameras in the coordinate system according to the target object, the first coordinate information at least including one of the coordinates of the target object in the coordinate system and the size of the target object; and determining whether the corresponding at least two cameras need to be corrected according to at least the first coordinate information or the second coordinate information. The method of the present disclosure can realize online detection of the camera.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more particularly to a camera detection method, apparatus, electronic device, and vehicle. Background Technology

[0002] Imaging systems are used in various scenarios, such as the 360-degree panoramic imaging (VAM) system in autonomous vehicles. The cameras in these systems may experience angular and positional shifts or deviations under external forces, necessitating the detection of these deviations for recalibration. Currently, testing the cameras in imaging systems requires laboratory testing or the use of objects of specific shapes and sizes for inspection. Summary of the Invention

[0003] This disclosure provides a camera detection method, apparatus, electronic device, and vehicle to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this disclosure, a camera detection method is provided, the method comprising:

[0005] Acquire images captured by each camera at the same time, with each image partially overlapping with images captured by at least two other cameras;

[0006] The target object is determined based on the overlapping portion of the images, wherein the target object is the same object corresponding to at least two cameras;

[0007] Establish a coordinate system;

[0008] The first coordinate information of the target object in the coordinate system is calculated based on at least two of the corresponding cameras, or the second coordinate information of the target object in the coordinate system is calculated based on the target object. The first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object.

[0009] Whether at least two of the cameras need calibration is determined based on the first coordinate information or the second coordinate information.

[0010] In one possible implementation, establishing the coordinate system includes:

[0011] The origin of the coordinate system is determined with reference to the spatial position of each of the cameras, or with reference to the spatial position of the target object.

[0012] When the origin of the coordinate system is determined with reference to the spatial position of each of the cameras, the first coordinate information of the target object in the coordinate system is calculated according to at least two of the corresponding cameras;

[0013] When the origin of the coordinate system is determined with reference to the spatial position of the target object, the second coordinate information of at least two cameras in the coordinate system is calculated based on the target object.

[0014] In one possible implementation, calculating the first coordinate information of the target object in the coordinate system based on at least two corresponding cameras includes:

[0015] The first coordinate information of the target object is calculated based on the initial parameters of the camera and the coordinates of the camera in the coordinate system.

[0016] In one possible implementation, the target object is any object in the overlapping portion of the image, or the target object is the same light source determined based on the light and shadow relationship in the overlapping portion of the image.

[0017] In one possible implementation, determining whether at least two of the corresponding cameras need calibration based on the first coordinate information includes:

[0018] If the first coordinate information calculated from different cameras is the same, at least two of the corresponding cameras do not need to be calibrated; otherwise, at least one of the corresponding cameras needs to be calibrated.

[0019] In one possible implementation, calculating the second coordinate information of at least two of the cameras in the coordinate system based on the target object includes:

[0020] The coordinates of the camera in the coordinate system are calculated based on the initial parameters of the camera and the coordinates of the target object in the coordinate system.

[0021] Based on the coordinates of the corresponding camera in the coordinate system, the second coordinate information of each camera is determined, wherein the second coordinate information includes at least one of the relative position and relative distance of each camera.

[0022] In one possible implementation, determining whether at least two of the corresponding cameras need to be calibrated based on the second coordinate information includes: if the calculated second coordinate information of each camera is the same as the initial coordinate information, the corresponding camera does not need to be calibrated; otherwise, at least one corresponding camera needs to be calibrated, wherein the initial coordinate information is the coordinate information of the camera after the previous calibration.

[0023] In one possible implementation, determining whether at least two of the corresponding cameras need calibration based on at least the first coordinate information or the second coordinate information includes:

[0024] If at least one of the corresponding at least two cameras needs to be calibrated, acquire an object of known size in the image captured by the camera that may need to be calibrated;

[0025] Based on the relationship between the known-sized object and the corresponding camera, determine whether the camera needs calibration.

[0026] In one possible implementation, determining whether at least two of the corresponding cameras need calibration based on at least the first coordinate information or the second coordinate information includes:

[0027] If either the first or second coordinate information violates natural laws, it is determined that the corresponding camera needs to be corrected.

[0028] According to a second aspect of this disclosure, a camera calibration method is provided, the method comprising:

[0029] When it is determined that the camera needs to be calibrated using the method described in the embodiments of this disclosure, calibration is performed based on the current parameters of the camera, or the position of the camera is adjusted.

[0030] According to a third aspect of this disclosure, a camera detection device is provided, the device comprising:

[0031] The acquisition module is used to acquire images captured by each camera at the same time, wherein the image captured by each camera partially overlaps with the images captured by at least two other cameras.

[0032] A determination module is used to determine a target object based on the overlapping portion of the image, wherein the target object is the same object corresponding to at least two cameras;

[0033] The module is used to establish a coordinate system;

[0034] The calculation module is used to calculate the first coordinate information of the target object in the coordinate system according to the corresponding at least two cameras, or to calculate the second coordinate information of the corresponding at least two cameras in the coordinate system according to the target object. The first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object.

[0035] The determination module is used to determine whether the corresponding camera needs to be calibrated, based at least on the first coordinate information or the second coordinate information.

[0036] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0037] At least one processor; and

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0040] According to a fourth aspect of this disclosure, a vehicle is provided that includes the electronic equipment of this disclosure.

[0041] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0042] In the camera detection method disclosed herein, images captured by each camera at the same time are acquired, thereby ensuring that the cameras can capture the same object in the same space and time. The target object is determined based on the overlapping portion of the images, ensuring that at least two cameras corresponding to the overlapping portion capture the same object, thus satisfying the condition of the same object. A coordinate system is established to transform the cameras with overlapping image portions and the objects they capture to the same coordinate system, satisfying the condition of the same space. First coordinate information of the target object in the coordinate system is calculated based on the corresponding at least two cameras, or second coordinate information of the corresponding at least two cameras in the coordinate system is calculated based on the target object. Based on the uniqueness of the same object in the same space and time, it is possible to determine whether at least two corresponding cameras need correction based on at least the first coordinate information or the second coordinate information. The method of this disclosure allows for online camera detection without a laboratory setting. It detects cameras by identifying the target object through the overlapping portion of images captured by different cameras, eliminating the need to pre-set objects of specific shapes and sizes, thus offering convenience and flexibility.

[0043] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0044] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0045] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0046] Figure 1 A schematic diagram illustrating the implementation flow of the camera detection method according to an embodiment of this disclosure is shown;

[0047] Figure 2 This illustration shows the principle of the camera detection method according to an embodiment of the present disclosure. Figure 1 ;

[0048] Figure 3 This illustration shows the principle of the camera detection method according to an embodiment of the present disclosure. Figure 2 ;

[0049] Figure 4 A schematic diagram of the composition structure of a camera detection device according to an embodiment of the present disclosure is shown;

[0050] Figure 5 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0051] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0052] This disclosure provides a camera detection method, see [link to relevant documentation]. Figure 1 The method includes:

[0053] Acquire images captured by each camera at the same time, with each image partially overlapping with images captured by at least two other cameras;

[0054] The target object is determined based on the overlapping parts of the images, and the target object is the same object corresponding to at least two cameras;

[0055] Establish a coordinate system;

[0056] The first coordinate information of the target object in the coordinate system is calculated based on at least two corresponding cameras, or the second coordinate information of the target object in the coordinate system is calculated based on the target object. The first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object.

[0057] Determine whether at least two corresponding cameras need calibration based on the first or second coordinate information.

[0058] In this disclosed camera detection method, images captured by various cameras at the same time are acquired, ensuring that the cameras can capture the same object in the same space-time. The target object is determined based on the overlapping portion of the images, ensuring that at least two cameras corresponding to the overlapping portion capture the same object, thus satisfying the condition of the same object. A coordinate system is established to transform the cameras with overlapping images and their captured objects to the same coordinate system, satisfying the condition of the same space. The first coordinate information of the target object in the coordinate system is calculated based on the corresponding at least two cameras, or the second coordinate information of the corresponding at least two cameras is calculated based on the target object. Based on the uniqueness of the same object in the same space-time, it is possible to determine whether at least two corresponding cameras need correction based on at least the first or second coordinate information. The method of this disclosure allows for online camera detection without a laboratory setting. It detects cameras by identifying the target object through the overlapping portion of images captured by different cameras, eliminating the need to pre-set objects of specific shapes and sizes, thus offering convenience and flexibility.

[0059] For some imaging systems, multiple cameras are required to achieve image acquisition without blind spots, and the areas captured by these cameras partially overlap. For example, a 360-degree panoramic imaging system (VAM) typically uses 4-8 cameras, and through video synthesis processing, forms a bird's-eye view, providing a 360-degree panoramic fusion, ultra-wide field of view, and seamless image stitching. When used in vehicles, this provides the driver with a wider field of vision and improves driving safety. The camera detection method of this disclosure can be used for camera detection in 360-degree panoramic imaging systems. It enables online camera detection without the need for pre-set markers, promptly identifying camera correction needs and allowing for timely camera calibration to improve imaging accuracy.

[0060] In this embodiment of the disclosure, calibrating the camera includes calibrating the camera's parameters and adjusting the camera's position and / or angle.

[0061] In this embodiment, a coordinate system is established to include all cameras and the target object within the same coordinate system. Based on the information of the cameras in the coordinate system, the coordinates of the object captured by each camera can be calculated within that coordinate system. Similarly, based on the information of the object in the coordinate system, the coordinates of the camera capturing that object can be calculated within that coordinate system. The coordinates of the same object within the same coordinate system are unique at any given time. Therefore, the coordinates of the same target object obtained from different cameras should be identical, or the relative coordinates of the same target object obtained from different cameras should be identical to the actual coordinates. Otherwise, the corresponding camera can be considered to require calibration. The coordinate information includes the object's coordinates, size, distance, etc., within the coordinate system. The established coordinate system can be a world coordinate system, which represents the absolute coordinates of the objective world.

[0062] In one possible implementation, establishing the coordinate system includes: determining the origin of the coordinate system with reference to the spatial positions of each camera; and, when the origin of the coordinate system is determined with reference to the spatial positions of each camera, calculating the first coordinate information of the target object in the coordinate system based on at least two corresponding cameras. The world coordinates can be determined based on the imaging system. The origin of the coordinate system is determined with reference to the spatial positions of each camera. Based on the relationship between the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, the coordinates of each camera in the imaging system in the world coordinate system, as well as their orientation relative to the world coordinate system, can be determined. Furthermore, the coordinates and dimensions of the object captured by the camera in the world coordinate system can be calculated. Taking a vehicle-mounted imaging system as an example, the origin can be the center of the rear axle of the vehicle, with the z-axis pointing upwards perpendicular to the ground through the roof, the y-axis pointing forward in the direction of travel, and the x-axis pointing to the right side of the vehicle when the vehicle is facing forward.

[0063] In specific implementation, see Figure 2 and Figure 3 The cameras capturing the target object include a first camera 1 and a second camera 2. The target object's first coordinates in the world coordinate system can be calculated based on the coordinates of the first camera 1 and the second camera 2. These first coordinates include the target object's coordinates or dimensions. If the positions or orientations of the first and second cameras remain unchanged, the first coordinates calculated by the first and second cameras should be consistent. Figure 2 As shown, if the first coordinate information, such as the position and size of the target object in the world coordinate system, calculated by camera 1 and camera 2 respectively, is consistent, and the target object is located at the position indicated by label 3, then camera 1 and camera 2 do not require calibration. If they are inconsistent, such as... Figure 3As shown, the first coordinate information of the target object in the world coordinate system calculated by camera 1 and the first coordinate information of the target object in the world coordinate system calculated by camera 2 are inconsistent. Specifically, the target object calculated by camera 1 is located at the position shown in label 3, and the target object calculated by camera 12 is located at the position shown in label 3'. Therefore, the position and / or orientation of at least one of the first camera 1 and the second camera 2 has changed and needs to be corrected.

[0064] In one possible implementation, the target object includes objects that overlap with the image and associated objects that can be determined by the objects that overlap.

[0065] In one possible implementation, determining a target object based on overlapping portions of images includes: determining the overlapping portions of at least two cameras based on the camera's viewing angle and mounting angle, and determining the target object from the portion of the overlapping portion that is furthest from the camera. Specifically, one object may be selected from objects in the overlapping portions furthest from the camera as the target object. Alternatively, related objects may be determined as the target object from objects in the overlapping portions furthest from the camera; for example, a corresponding light source may be determined based on the objects in the overlapping portions furthest from the camera and their shadows.

[0066] In one possible implementation, establishing a coordinate system includes: determining the origin of the coordinate system with reference to the spatial position of the target object; and, when the origin of the coordinate system is determined with reference to the spatial position of the target object, calculating the second coordinate information of at least two corresponding cameras in the coordinate system based on the target object. The world coordinates can also be determined based on the captured object. According to the relationship between the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, the coordinate information of the captured object in the world coordinate system can be determined, and the coordinate information of the corresponding cameras in the world coordinate system can be calculated accordingly. For example, the world coordinate system can have the center of the target object as its origin, and the specific directions of the z-axis, y-axis, and x-axis can be set as needed.

[0067] In practice, the cameras capturing the target object include a first camera 1 and a second camera 2. Based on the target object's coordinates in the world coordinate system, the second coordinates of the first camera 1 and the second camera 2 in the world coordinate system can be calculated. The distance between the two cameras can be determined from their coordinates in the world coordinate system. If the calculated distance matches the known distance, no calibration is needed. If the calculated distance differs from the known distance, the position and / or orientation of at least one of the first camera 1 and the second camera 2 has changed, requiring calibration.

[0068] In one embodiment, calculating the first coordinate information of the target object in the coordinate system based on at least two corresponding cameras includes: calculating the first coordinate information of the target object based on the initial parameters of the cameras and the coordinates of the cameras in the coordinate system. In this embodiment, the initial parameters of the cameras include the camera angle. Taking a vehicle imaging system as an example, the origin of the established world coordinate system is a point related to the vehicle, for example, the center of the rear axle of the vehicle is the origin of the world coordinate system. Based on the position of the camera on the vehicle, a relationship between the world coordinate system and the camera coordinate system can be established, [R, T] where T is the translation of the origin of the camera coordinate system relative to the origin of the world coordinate system, and R is the rotation matrix of the camera coordinate system relative to the world coordinate system. The coordinates and orientation of the camera in the world coordinate system can be obtained through the relationship between the world coordinate system and the camera coordinate system. Based on the transformation of the pixel coordinate system and the transformation of pinhole imaging, the first coordinate information of the target object in the world coordinate system can be determined. Taking a rectangular sign as an example, the coordinates of its four corners in the world coordinate system can be determined, and the length and width of the sign can be obtained. If the coordinates or dimensions of the sign are different based on the first camera 1 and the second camera 2, then the position or angle of at least one of the first camera 1 and the second camera 2 has changed and needs correction. In this case, the above method can be continued: the first coordinate information of the target object can be determined jointly by a third camera with an overlapping area with the image captured by the second camera 2. If the first coordinate information of the target object obtained by the third camera and the second camera 2 is consistent, then it can be determined that the second camera 2 and the third camera do not need correction. Therefore, it can be determined that the first camera 1 needs correction. For example, the target object corresponding to the second camera 2 and the third camera could be a vehicle. The second camera 2 and the third camera can respectively obtain the vehicle's first coordinate information, such as its coordinates and length in the world coordinate system.

[0069] In one possible implementation, the target object can be any object within the overlapping portion of the images, or the target object can be the same light source determined based on the light and shadow relationships within the overlapping portion. The target object is determined based on the overlapping portion of the images; it can be selected from objects within the overlapping portion, or it may not be in the overlapping portion, or even in any image. However, the position of the target image can be determined based on the overlapping portion, for example, by determining the corresponding light source based on the light and shadow relationships. Based on an object and its shadow within the overlapping portion of the images, the position of the light source forming the shadow can be obtained. The same object and its shadow captured by different cameras at the same time correspond to the same light source, and the coordinate information of this light source in the world coordinate system should be unique. Based on this, it can be determined whether the corresponding camera needs calibration. The light source can be a street lamp, the sun, etc.

[0070] In one possible implementation, determining whether at least two corresponding cameras need calibration based on first coordinate information includes: if the first coordinate information calculated from different cameras is the same, the at least two corresponding cameras do not need calibration; otherwise, at least one corresponding camera needs calibration. If the first coordinate information is the same, it satisfies the uniqueness of the same object in the same coordinate system within the same image at the same time, and the position and angle of the corresponding camera capturing the image do not need adjustment, nor does it require recalibrating the camera parameters. If the first information obtained from different cameras is different, it indicates that at least one camera has deviated, and the parameters need to be recalibrated or the position and / or angle of the camera needs to be adjusted.

[0071] In one embodiment, calculating the second coordinate information of at least two corresponding cameras in a coordinate system based on the target object includes: calculating the coordinates of the corresponding cameras in the coordinate system based on the initial parameters of the cameras and the coordinates of the target object; and determining the second coordinate information of each camera based on the coordinates of the corresponding cameras in the coordinate system. The second coordinate information includes at least one of the relative position and relative distance of each camera. The initial parameters of the cameras are the parameters previously calibrated by the cameras. The first coordinate information of the target object obtained from different cameras or the second coordinate information of the corresponding different cameras obtained from the target object are both calculated based on the initial parameters.

[0072] In one possible implementation, determining whether at least two corresponding cameras need calibration based on second coordinate information includes: if the calculated second coordinate information of each camera is the same as the initial coordinate information, the corresponding camera does not need calibration; otherwise, at least one corresponding camera needs calibration. The initial coordinate information is the coordinate information of the camera after its previous calibration. If the distance or position of the corresponding camera calculated based on the target object does not match the initial coordinate information, it can be determined that at least one of the corresponding cameras needs calibration. The relative positions and distances of the cameras in the imaging system are known as initial coordinate information. Whether a camera needs calibration can be determined by whether the calculated second coordinate information is consistent with the known initial coordinate information. For example, a vehicle imaging system includes eight cameras arranged around the circumference of the vehicle. The positions and distances of each camera are known. Along the circumference of the vehicle, images captured by adjacent cameras have overlapping portions, and the images captured by the eight cameras can be combined to form a bird's-eye view of the vehicle's surroundings. By verifying each pair of cameras against each other, it can be determined which cameras need calibration.

[0073] In one possible implementation, determining whether at least two corresponding cameras need calibration based on at least first coordinate information or second coordinate information includes: if at least one of the at least two corresponding cameras needs calibration, acquiring an object of known size in the image captured by the camera that may need calibration; and determining whether the camera needs calibration based on the relationship between the object of known size and the corresponding camera. If it is determined that at least one of the at least two corresponding cameras needs calibration, auxiliary verification can be performed using an object of known size in the image. If the size determined by the camera matches the known size of the object, then the camera does not need calibration; otherwise, the camera needs calibration. For example, if the images captured by the first camera 1 and the second camera 2 have overlapping portions, and the first coordinate information of the target object calculated by the first camera 1 and the second camera 2 is inconsistent, or the second coordinate information calculated by the first camera 1 and the second camera 2 based on the target object is inconsistent with the initial coordinate information, then it is determined that at least one of the first camera 1 and the second camera 2 needs correction. In this case, an object of known size can be obtained from the image captured by the first camera 1, such as a license plate. License plates are manufactured according to standard sizes, such as 440 mm * 140 mm for a small car license plate. Through the transformation of the coordinate system of the first camera 1, the transformation of the pixel coordinate system, and the transformation of the pinhole imaging, the size of the license plate can be calculated. If the calculated size of the license plate does not match the actual size, then the first camera 1 needs correction. At this time, it is also necessary to perform auxiliary verification on the second camera 2 to determine whether correction is needed. If the calculated size of the license plate matches the actual size, then the first camera 1 does not need correction. Since at least one of the first camera 1 and the second camera 2 needs correction, if it is determined that the first camera 1 does not need correction, then the second camera 2 needs correction. Similarly, the same method can be used to perform auxiliary verification on the second camera 2 to determine whether correction is needed. The objects of known size are not limited to overlapping parts; any object in the image captured by the camera is acceptable.

[0074] In one possible implementation, determining whether at least two corresponding cameras need calibration is based on at least first coordinate information or second coordinate information includes: determining that the corresponding camera needs calibration if the first or second coordinate information violates natural laws. If the calculated first or second coordinate information violates natural laws, it indicates that the first or second coordinate information calculated based on the camera's initial parameters is incorrect, and the camera needs calibration. Violating natural laws includes being inconsistent with objective facts, including, according to the first coordinate information, the corresponding target object being located in an impossible position. For example, if the target object should be on the ground, but according to the first coordinate information, the target object is suspended in the air or embedded in the ground, this violates natural laws, and the corresponding camera needs calibration. In an exemplary embodiment, the images captured by the first camera 1 and the second camera 2 have overlapping portions, and the target object is a trash can. Since the first coordinate information of the target object calculated by the first camera 1 and the second camera 2 is inconsistent, it is determined that at least one of the first camera 1 and the second camera 2 needs to be corrected. The first coordinate information of the target object calculated by the first camera 1 indicates that the target object is suspended in the air, while the trash can should be on the ground. Therefore, the first coordinate information of the target object calculated by the first camera 1 violates the laws of nature, and the first camera 1 needs to be corrected.

[0075] Defying the laws of nature also includes situations where, according to the information in the second coordinate system, the corresponding camera is located in an impossible position. For example, if all the cameras in a vehicle's imaging system are located on the vehicle itself, but according to the second coordinate information, the corresponding camera is located outside the vehicle body, then this violates the laws of nature.

[0076] This disclosure provides a camera calibration method, which includes: calibrating the camera based on its current parameters when the method of this disclosure determines that the camera needs calibration. In an exemplary embodiment, the difference between the first coordinate information or the second coordinate information of the same target object obtained by the camera requiring calibration and the camera not requiring calibration can be used to obtain the deviation of the camera requiring calibration. Based on this deviation and the original parameters of the camera requiring calibration, the current parameters of the camera can be obtained, thereby enabling camera calibration. For example, if the images captured by the first camera 1 and the second camera 2 have overlapping portions, and the images captured by the second camera 2 and the third camera have overlapping portions, the first coordinate information or the second coordinate information obtained from the overlapping portions of the images captured by the first camera 1 and the second camera 2 determines that at least one of the first camera 1 and the second camera 2 needs calibration. The first coordinate information or the second coordinate information obtained from the overlapping portions of the images captured by the third camera and the second camera 2 determines that the third camera and the second camera 2 do not need calibration. Therefore, it can be finally determined that the first camera 1 needs calibration. Based on the first coordinate information or the second coordinate information obtained by the first camera 1 and the second camera 2 respectively, the deviation of the first camera 1 requiring calibration can be obtained. Based on this deviation and the original parameters of the first camera 1, the current parameters of the first camera 1 can be obtained.

[0077] Alternatively, if it is determined that the camera needs calibration, the camera's position and / or angle can be adjusted. If the camera's position or angle is significantly off, adjusting the camera's position and / or angle can be used when parameter calibration alone cannot achieve camera calibration. When adjusting the camera's position or angle, adjustments can also be made based on the deviation of the camera requiring calibration.

[0078] This disclosure provides a camera detection device, see [link to relevant documentation]. Figure 4 The device includes an acquisition module, a determination module, an establishment module, a calculation module, and a judgment module. The acquisition module is used to acquire images captured by each camera at the same time, wherein the image captured by each camera partially overlaps with the images captured by at least two other cameras. The determination module is used to determine the target object based on the overlapping portion of the images, wherein the target object is the same object corresponding to at least two cameras. The establishment module is used to establish a coordinate system. The calculation module is used to calculate the first coordinate information of the target object in the coordinate system based on the corresponding at least two cameras, or to calculate the second coordinate information of the corresponding at least two cameras in the coordinate system based on the target object, wherein the first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object. The judgment module is used to determine whether the corresponding camera needs to be calibrated based on at least the first coordinate information or the second coordinate information.

[0079] In the camera detection device disclosed herein, the acquisition module acquires images captured by each camera at the same time, thereby ensuring that the cameras can capture the same object in the same space-time; the determination module determines the target object based on the overlapping portion of the images, ensuring that at least two cameras corresponding to the overlapping portion capture the same object, thus satisfying the condition of the same object; the establishment module establishes a coordinate system to transform the cameras with overlapping image portions and the objects they capture to the same coordinate system, satisfying the condition of the same space; the calculation module calculates the first coordinate information of the target object in the coordinate system based on the corresponding at least two cameras, or calculates the second coordinate information of the corresponding at least two cameras in the coordinate system based on the target object. Based on the uniqueness of the same object in the same space-time, the determination module can determine whether the corresponding at least two cameras need correction based on at least the first coordinate information or the second coordinate information. The device of this disclosure embodiment can perform online camera detection without a laboratory setting. It detects cameras by determining the target object through the overlapping portion of images captured by different cameras, and can detect cameras without pre-setting objects of specific shapes and sizes, which is convenient and flexible.

[0080] In one possible implementation, the establishment of the coordinate system by the module includes: determining the origin of the coordinate system with reference to the spatial position of each camera; and, when the origin of the coordinate system is determined with reference to the spatial position of each camera, calculating the first coordinate information of the target object in the coordinate system based on at least two corresponding cameras.

[0081] In one possible implementation, the target object includes objects that overlap with the image and associated objects that can be determined by the objects that overlap.

[0082] In one possible implementation, the determining module determines the target object based on the overlapping portion of the images, including: determining the overlapping portion of at least two cameras based on the camera's viewing angle and installation angle, and determining the target object from the portion of the overlapping portion that is far away from the camera.

[0083] In one implementation, establishing a coordinate system by the module includes: determining the origin of the coordinate system with reference to the spatial position of the target object; and, when the origin of the coordinate system is determined with reference to the spatial position of the target object, calculating the second coordinate information of at least two corresponding cameras in the coordinate system based on the target object. World coordinates can also be determined based on the captured object. According to the relationship between the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, the coordinate information of the captured object in the world coordinate system can be determined, and the coordinate information of the corresponding camera in the world coordinate system can be calculated accordingly.

[0084] In one embodiment, the calculation module calculates the first coordinate information of the target object in the coordinate system based on at least two corresponding cameras, including: calculating the first coordinate information of the target object based on the initial parameters of the cameras and the coordinates of the cameras in the coordinate system.

[0085] In one possible implementation, the target object is any object in the overlapping portion of the image, or the target object is the same light source determined based on the light and shadow relationship in the overlapping portion of the image.

[0086] In one possible implementation, the determination module determines whether at least two corresponding cameras need to be calibrated based on the first coordinate information, including: if the first coordinate information calculated based on different cameras is the same, at least two corresponding cameras do not need to be calibrated; otherwise, at least one of the corresponding cameras needs to be calibrated.

[0087] In one embodiment, the calculation module calculates the second coordinate information of at least two corresponding cameras in a coordinate system based on the target object, including: calculating the coordinates of the corresponding cameras in the coordinate system based on the initial parameters of the cameras and the coordinates of the target object in the coordinate system; and determining the second coordinate information of each camera based on the coordinates of the corresponding cameras in the coordinate system, wherein the second coordinate information includes at least one of the relative position and relative distance of each camera.

[0088] In one possible implementation, the calculation module determines whether at least two corresponding cameras need to be calibrated based on the second coordinate information, including: if the calculated second coordinate information of each camera is the same as the initial coordinate information, the corresponding camera does not need to be calibrated; otherwise, at least one corresponding camera needs to be calibrated, and the initial coordinate information is the coordinate information of the camera after the previous calibration.

[0089] In one possible implementation, the determination module determines whether at least two corresponding cameras need to be calibrated based on at least the first coordinate information or the second coordinate information, including: if at least one of the at least two corresponding cameras needs to be calibrated, acquiring an object of known size in the image captured by the camera that may need to be calibrated; and determining whether the camera needs to be calibrated based on the relationship between the object of known size and the corresponding camera.

[0090] In one possible implementation, the determination module determines whether at least two corresponding cameras need calibration based on at least the first coordinate information or the second coordinate information. This includes determining that the corresponding camera needs calibration if the first or second coordinate information violates natural laws. Violating natural laws also includes, according to the second coordinate information, the corresponding camera being located in an impossible position. For example, if all cameras in a vehicle's imaging system are mounted on the vehicle, but according to the second coordinate information, the corresponding camera is located outside the vehicle body, this violates natural laws.

[0091] The camera detection device of this disclosure can implement the methods of the above embodiments. The description of the camera detection device embodiments above is similar to the description of the foregoing camera detection method embodiments, and has similar beneficial effects as the foregoing method embodiments, so it will not be described again. For technical details not disclosed in the description of the camera detection device of this disclosure, please refer to the description of the foregoing method embodiments of this disclosure for understanding, and for the sake of saving space, it will not be described again.

[0092] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium. The electronic device of this disclosure includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0093] According to embodiments of this disclosure, this disclosure also provides a vehicle that includes electronic devices according to embodiments of this disclosure.

[0094] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0095] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0096] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the camera detection method. For example, in some embodiments, the camera detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the camera detection method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the camera detection method by any other suitable means (e.g., by means of firmware).

[0098] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0102] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0103] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0104] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0105] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0106] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A camera detection method, characterized in that, The method includes: Acquire images captured by each camera at the same time, with each image partially overlapping with images captured by at least two other cameras; The target object is determined based on the overlapping portion of the images, wherein the target object is the same object corresponding to at least two cameras; Establish a coordinate system: Determine the origin of the coordinate system with reference to the spatial position of each camera, or determine the origin of the coordinate system with reference to the spatial position of the target object; When the origin of the coordinate system is determined with reference to the spatial position of each of the cameras, the first coordinate information of the target object is calculated based on the initial parameters of at least two of the corresponding cameras and the coordinates of the cameras in the coordinate system; when the origin of the coordinate system is determined with reference to the spatial position of the target object, the coordinates of at least two of the corresponding cameras in the coordinate system are calculated based on the initial parameters of at least two of the corresponding cameras and the coordinates of the target object in the coordinate system; based on the coordinates of at least two of the corresponding cameras in the coordinate system, the second coordinate information of each camera is determined, the second coordinate information including at least one of the relative position and relative distance of each camera; the first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object; If the first coordinate information calculated from different cameras is different, or the second coordinate information calculated from the target object is different from the coordinate information of the camera after the previous calibration, or if the first coordinate information or the second coordinate information violates natural laws, it is determined that at least one of the corresponding at least two cameras needs calibration; wherein, the first coordinate information or the second coordinate information violating natural laws includes: according to the first coordinate information, the corresponding target object is located in an impossible position; or according to the second coordinate information, the corresponding camera is located in an impossible position; if at least one of the corresponding at least two cameras needs calibration, an object of known size in the image captured by the camera that may need calibration is obtained; based on the relationship between the object of known size and the corresponding camera, it is determined whether the camera needs calibration.

2. The method according to claim 1, characterized in that, The target object can be any object in the overlapping portion of the image, or the target object can be the same light source determined based on the light and shadow relationship in the overlapping portion of the image.

3. A camera calibration method, characterized in that, When it is determined that the camera needs to be calibrated using the method described in any one of claims 1-2, the camera is calibrated based on its current parameters, or the position of the camera is adjusted.

4. A camera detection device, characterized in that, The device includes: The acquisition module is used to acquire images captured by each camera at the same time, wherein the image captured by each camera partially overlaps with the images captured by at least two other cameras. A determination module is used to determine a target object based on the overlapping portion of the image, wherein the target object is the same object corresponding to at least two cameras; A module is established to establish a coordinate system: the origin of the coordinate system is determined with reference to the spatial position of each of the cameras, or with reference to the spatial position of the target object. The calculation module is configured to: calculate first coordinate information of the target object based on the initial parameters of at least two of the corresponding cameras and the coordinates of the cameras in the coordinate system, when the origin of the coordinate system is determined with reference to the spatial position of the target object; calculate the coordinates of at least two of the corresponding cameras in the coordinate system based on the initial parameters of the at least two corresponding cameras and the coordinates of the target object in the coordinate system; and determine second coordinate information of each camera based on the coordinates of the at least two corresponding cameras in the coordinate system, wherein the second coordinate information includes at least one of the relative position and relative distance of each camera; the first coordinate information includes at least one of the coordinates of the target object in the coordinate system and the size of the target object. The judgment module is used to determine that at least one of the at least two cameras needs correction when the first coordinate information calculated based on different cameras is different, or the second coordinate information calculated based on the target object is different from the coordinate information of the camera after the previous correction, or when the first coordinate information or the second coordinate information violates natural laws; wherein, the first coordinate information or the second coordinate information violates natural laws including: according to the first coordinate information, the corresponding target object is located in an impossible position; or according to the second coordinate information, the corresponding camera is located in an impossible position; when at least one of the at least two cameras needs correction, the module obtains an object of known size in the image captured by the camera that may need correction; and determines whether the camera needs correction based on the relationship between the object of known size and the corresponding camera.

5. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-2.

6. A vehicle, characterized in that, Includes the electronic device as described in claim 5.

7. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method according to any one of claims 1-2.

Images