Multi-lens image mapping method, device, chip, storage medium and program product

Abstract

This application provides a multi-lens image mapping method, apparatus, chip, storage medium, and program product. The method includes: acquiring lens calibration data; wherein the lens calibration data includes calibration center point offset, calibration distance, and field-of-view ratio information; the calibration center point offset is used to indicate the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; acquiring the position information and ranging information of the target object in the main image; and calculating the mapped position information of the target object in the auxiliary image based on the ranging information, lens calibration data, and position information, wherein the auxiliary image is the image of the target object acquired by the auxiliary lens. This method can reduce computing power consumption and device load, extend device battery life, and avoid device overheating.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a multi-lens image mapping method, apparatus, chip, storage medium and program product. Background Technology

[0002] With the increasing prevalence of multi-camera devices, the demand for multi-lens collaborative operation in terminal products is growing. In real-time shooting or recording scenarios, the main image is used for user preview, while the auxiliary images are used to generate high-quality images.

[0003] Currently, multi-lens devices typically use separate face detection algorithms to process the main road image and the auxiliary road image respectively.

[0004] However, existing technologies require twice the computing resources for face detection, leading to increased device load, shorter battery life, and potential overheating issues. Summary of the Invention

[0005] This application provides a multi-lens image mapping method, apparatus, chip, storage medium, and program product to reduce device load, extend battery life, and prevent device overheating.

[0006] In a first aspect, this application provides a multi-lens image mapping method, including:

[0007] Acquire lens calibration data; wherein the lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset is used to indicate the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance is used to indicate the horizontal distance between the calibration object and the main lens or the auxiliary lens; the field of view ratio information is used to indicate the ratio relationship between the field of view of the main lens and the auxiliary lens;

[0008] The location information and distance measurement information of the target object in the main road image are obtained, wherein the main road image is the image of the target object captured by the main road lens, and the distance measurement information is used to indicate the horizontal distance between the target object and the main road lens or the auxiliary road lens;

[0009] Based on the ranging information, the lens calibration data, and the position information, the mapping position information of the target object in the auxiliary path image is calculated, wherein the auxiliary path image is an image of the target object captured by the auxiliary path lens.

[0010] In one possible implementation, calculating the mapping position information of the target object in the auxiliary road image based on the ranging information, the lens calibration data, and the position information includes: calculating the target center point offset based on the lens calibration data and the ranging information, wherein the target center point offset is used to indicate the coordinate offset of the image center point of the target object captured by the auxiliary road lens relative to the image center point of the target object captured by the main road lens; and calculating the mapping position information of the target object in the auxiliary road image based on the position information, the target center point offset, and the field of view ratio information.

[0011] In one possible implementation, the step of calculating the target center point offset based on the lens calibration data and the ranging information includes: parsing the lens calibration data to obtain the calibration center point offset and the calibration distance; and calculating the target center point offset using the principle of similar triangles based on the calibration center point offset, the calibration distance, and the ranging information.

[0012] In one possible implementation, calculating the mapping position information of the target object in the auxiliary road image based on the position information, the target center point offset, and the field of view ratio information includes: offsetting the position information according to the target center point offset to obtain offset position information; scaling the offset position information according to the field of view ratio information to obtain scaled position information; obtaining the image setting size of the auxiliary road lens; and performing position compensation on the scaled position information according to the image setting size to obtain the mapping position information of the target object in the auxiliary road image.

[0013] In one possible implementation, before acquiring the position information and ranging information of the target object in the main road image, the method further includes: monitoring lens attitude changes through a built-in sensor to obtain lens attitude information; and updating the lens calibration data based on the lens attitude information.

[0014] In one possible implementation, the step of performing position compensation on the scaled position information according to the image set size to obtain the mapping position information of the target object in the auxiliary road image includes: using an edge detection algorithm to evaluate the sharpness of the main road image to obtain a sharpness quantification value; determining the position compensation accuracy based on the sharpness quantification value; and performing position compensation on the scaled position information according to the position compensation accuracy and the image set size to obtain the mapping position information of the target object in the auxiliary road image.

[0015] Secondly, this application provides a multi-lens image mapping device, comprising:

[0016] The first acquisition module is used to acquire lens calibration data; wherein the lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset is used to indicate the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance is used to indicate the horizontal distance between the calibration object and the main lens or the auxiliary lens; the field of view ratio information is used to indicate the multiple relationship between the field of view of the main lens and the auxiliary lens;

[0017] The second acquisition module is used to acquire the position information and distance information of the target object in the main road image, wherein the main road image is the image of the target object captured by the main road lens, and the distance information is used to indicate the horizontal distance between the target object and the main road lens or the auxiliary road lens;

[0018] The calculation module is used to calculate the mapping position information of the target object in the auxiliary road image based on the ranging information, the lens calibration data and the position information, wherein the auxiliary road image is the image of the target object captured by the auxiliary road lens.

[0019] Thirdly, this application provides a chip including at least one processor for executing program instructions to perform the first aspect and / or various possible implementations of the first aspect.

[0020] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect.

[0021] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0022] The multi-lens image mapping method, apparatus, chip, storage medium, and program product provided in this application calculate the mapped position information of the target object in the auxiliary path image using the position information of the target object in the main path image, along with lens calibration data and ranging information. This process eliminates the need to execute a separate detection algorithm on the auxiliary path image; instead, it performs mathematical calculations using the main path image and lens calibration data, thereby reducing computational power consumption and equipment load, extending equipment battery life, and preventing equipment overheating. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 This is an example image of face detection in the main and auxiliary road images in the existing technology;

[0025] Figure 2 A schematic diagram of a scene for the multi-lens image mapping method provided in an embodiment of this application;

[0026] Figure 3 A flowchart illustrating a multi-lens image mapping method provided in one embodiment of this application;

[0027] Figure 4 An example image of an object being photographed by a multi-lens device at two positions: a calibration distance and a ranging information, as provided in an embodiment of this application;

[0028] Figure 5 Example diagrams of images captured by the main path camera and auxiliary path camera provided in the embodiments of this application;

[0029] Figure 6 Example diagram of the main road image provided in the embodiments of this application;

[0030] Figure 7 Example diagram of auxiliary road image provided in the embodiments of this application;

[0031] Figure 8 The embodiments provided in this application will Figure 6 Example image after offset;

[0032] Figure 9 The embodiments provided in this application will Figure 8 Example image after scaling;

[0033] Figure 10 The embodiments provided in this application provide for the Figure 9 Example image after position compensation;

[0034] Figure 11 Example diagrams for face detection in main and auxiliary road images provided in this application;

[0035] Figure 12 This is a schematic diagram of the structure of the multi-lens image mapping device provided in the embodiments of this application;

[0036] Figure 13 This is a schematic diagram of the structure of a multi-lens device provided in an embodiment of this application.

[0037] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0040] With the increasing popularity of multi-lens devices (such as smartphones, drones, cameras, etc.), users' demand for image processing performance is growing.

[0041] Figure 1 This is an example image of face detection in the main and auxiliary road images of existing technology. (Example image follows) Figure 1 As shown, the detection process specifically includes: acquiring images through the main path wide-angle lens to generate the main path image; acquiring images through the auxiliary path ultra-wide-angle lens to generate the auxiliary path image; both images are multi-view / multi-focal length input sources acquired in parallel. A face detection algorithm is executed on the main path image to identify and locate face regions in the image; a face detection algorithm is simultaneously executed on the auxiliary path image to complete face region identification and localization from the auxiliary path perspective. Image processing algorithms are applied to the main path image after face detection; image processing algorithms are simultaneously applied to the auxiliary path image after face detection to complete corresponding preprocessing or enhancement operations. The processed results from both the main and auxiliary path images are input into a blurring algorithm, combining the face position, depth information, or viewpoint differences between the two images to achieve the final effect of focusing the target area (such as a face) and blurring the background.

[0042] These devices often need to process main and auxiliary images simultaneously during operation. If the main and auxiliary cameras acquire images at the same time, both images need to be processed by computationally intensive face detection algorithms, which leads to a significant increase in device power consumption, a shortened battery life, and may cause the device to overheat.

[0043] This application provides a multi-lens image mapping method. Addressing the high power consumption issue of existing technologies, the inventors explore how to optimize face detection in auxiliary path images using the geometric relationships of the lenses. First, by analyzing the lens calibration data and field-of-view ratio information of the main and auxiliary path lenses, a spatial correlation between the positions of target objects in the two images is discovered. Further, combining device ranging information, the mapped position information of the auxiliary path image is calculated using the target object position information in the main path image and the aforementioned parameters. This method avoids face detection in the auxiliary path image, reduces device load, extends battery life, and prevents device overheating.

[0044] Figure 2 This is a scene illustration of the multi-lens image mapping method provided in the embodiments of this application, such as... Figure 2 As shown, the specific application scenarios of this application include: multi-lens device 20.

[0045] The multi-lens device 20 includes multiple lenses with different focal lengths, which can be a main lens 201 and an auxiliary lens 202.

[0046] Among them, the multi-lens device 20 can be a device that includes multiple lenses with different focal lengths, including mobile phones, drones or cameras.

[0047] The device that performs the multi-lens image mapping method in the embodiments of this application can be a multi-lens device 201, or it can be another device independent of the multi-lens device 20. This application does not limit this.

[0048] The following explanation will take a multi-lens device 20 as the execution subject as an example.

[0049] Specifically, the multi-lens device 20 acquires lens calibration data, position information of the target object in the main road image, and ranging information; based on the ranging information, lens calibration data, and position information, the multi-lens device 20 calculates the mapping position information of the target object in the auxiliary road image.

[0050] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0051] Figure 3 This is a flowchart illustrating a multi-lens image mapping method provided in one embodiment of this application. The execution entity of this embodiment can be... Figure 2 The multi-lens device 20 shown, or the terminal device / base station, or the chip or chip module in the terminal device / base station, can also be other computer devices; this embodiment does not impose any particular limitations on this. Figure 3As shown, the method includes:

[0052] S301: Acquire lens calibration data; wherein the lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset is used to indicate the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance is used to indicate the horizontal distance between the calibration object and the main lens or auxiliary lens; the field of view ratio information is used to indicate the ratio of the field of view of the main lens and the auxiliary lens.

[0053] Optionally, the main path lens can be a wide-angle lens; the auxiliary path lens can be an ultra-wide-angle lens.

[0054] Optionally, the calibration center point offset can be the lens physical property parameters recorded by the multi-lens device through calibration experiments, including calibration center point offset, calibration distance, and field of view ratio information.

[0055] S302: Obtain the position information and distance information of the target object in the main road image, wherein the main road image is the image of the target object captured by the main road camera, and the distance information is used to indicate the horizontal distance between the target object and the main road camera or the auxiliary road camera.

[0056] Optionally, the ranging information can be the real-time horizontal distance between the target object and the main road camera or auxiliary road camera, obtained through the ranging device.

[0057] Optionally, the location information includes the coordinates of the top-left point of the target object's bounding box and the size information of the bounding box.

[0058] Optionally, obtaining the location information of the target object in the main road image includes: performing detection processing on the main road image using an image processing algorithm to obtain the location information of the target object.

[0059] S303: Based on the ranging information, lens calibration data, and position information, the mapping position information of the target object in the auxiliary road image is calculated, where the auxiliary road image is the image of the target object captured by the auxiliary road lens.

[0060] Optionally, based on ranging information, lens calibration data, and position information, the mapped position information of the target object in the auxiliary road image is calculated, including: calculating the target center point offset based on lens calibration data and ranging information, wherein the target center point offset is used to indicate the coordinate offset of the image center point of the target object captured by the auxiliary road lens relative to the image center point of the target object captured by the main road lens; and calculating the mapped position information of the target object in the auxiliary road image based on position information, target center point offset, and field of view ratio information.

[0061] By first calculating the coordinate offset of the target center point between the main and auxiliary road lenses, and then combining the position information and the field of view ratio to calculate the mapping position, the mapping position of the target object in the auxiliary road image can be obtained more accurately and stably, reducing the target position mapping error between the main and auxiliary roads and improving the accuracy of the mapping calculation.

[0062] Optionally, the target center point offset is calculated based on lens calibration data and ranging information, including: calculating the target center point offset using the principle of similar triangles based on lens calibration data and ranging information.

[0063] Optionally, the target center point offset is calculated based on the lens calibration data and ranging information, including: parsing the lens calibration data to obtain the calibration center point offset and calibration distance; and calculating the target center point offset using the principle of similar triangles based on the calibration center point offset, calibration distance, and ranging information.

[0064] By dynamically calculating the target center point offset using the principle of similar triangles, the mapping error problem caused by long-term equipment use or environmental changes in traditional static calibration is solved. Mathematical relationships are used to accurately compensate for geometric differences between lenses, ensuring that the position calculation results of the auxiliary lens image are aligned with the main lens image, reducing equipment load and extending equipment battery life.

[0065] Figure 4 This image illustrates an example of an object captured by a multi-lens device at two locations: a calibration distance and a ranging information location, as provided in an embodiment of this application. The red and yellow dots in the image represent the center points of the longitudinal section 1 and longitudinal section 2 images captured by the main lens and auxiliary lens at the calibration distance and ranging information locations, respectively. Figure 4 As shown in the figure, the field of view of the multi-lens device is displayed. m1 is the calibration distance, m2 is the ranging information. At the calibration distance, the lateral offset of the red and yellow dots is dx1, and the longitudinal offset is dy1.

[0066] Optionally, by Figure 4 It can be seen that the formulas for calculating the target center point offset are: dx2=dx1*m1 / (m1+m2), dy2=dy1*m1 / (m1+m2). In these formulas, dx2 is the lateral offset of the target center point offset; dy2 is the longitudinal offset of the target center point offset; m1 is the calibration distance; m2 is the distance measurement information; dx1 is the lateral offset of the calibration center point offset; and dy1 is the longitudinal offset of the calibration center point offset.

[0067] Optionally, the mapping position information of the target object in the auxiliary road image is calculated based on the position information, the target center point offset, and the field of view ratio information. This includes: offsetting the position information according to the target center point offset to obtain offset position information; scaling the offset position information according to the field of view ratio information to obtain scaled position information; obtaining the image setting size of the auxiliary road lens; and performing position compensation on the scaled position information according to the image setting size to obtain the mapping position information of the target object in the auxiliary road image.

[0068] Position information is offset by the target center point offset to avoid mapping errors caused by differences in lens position; through scaling and position compensation, the mapped position information of the auxiliary road image is ensured to match the position information of the main road image. This further improves the accuracy of the mapping results without requiring a detection algorithm, thereby reducing equipment load and extending equipment battery life.

[0069] Optionally, the offset position information includes the offset coordinates of the top-left point of the target object's bounding box and the offset size information of the bounding box.

[0070] Optionally, the offset dimension information of the rectangle includes the offset width and the offset height of the rectangle.

[0071] Optionally, the upper left point of the auxiliary road image and the main road image can be used as the origin, with the X-axis extending horizontally to the right and the Y-axis extending vertically downward.

[0072] Optionally, the formulas for calculating the offset coordinates of the top-left point and the offset dimensions of the rectangle are: x1=x0+dx2; y1=y0+dy2; w1=w0; h1=h0. Where x1, y1, w1, and h1 are the x-coordinate of the top-left offset of the target object's rectangle, the y-coordinate of the top-left offset of the rectangle, the offset width of the rectangle, and the offset height of the rectangle, respectively; and x0, y0, w0, and h0 are the x-coordinate of the top-left offset of the target object's rectangle detected in the main path image, the y-coordinate of the top-left offset of the rectangle, the width of the rectangle, and the height of the rectangle, respectively.

[0073] Optionally, the offset position information is scaled proportionally according to the field of view ratio information to obtain the scaled position information, including: obtaining the ratio of the field of view of the main road camera and the auxiliary road camera according to the field of view ratio information; and scaling the offset position information proportionally according to the ratio to obtain the scaled position information.

[0074] Optionally, the scaling position information includes the scaling coordinates of the top-left point of the target object's bounding box and the scaling size information of the bounding box.

[0075] Optionally, the scaling size information of the rectangle includes the scaling width and scaling height of the rectangle.

[0076] Optionally, the formulas for calculating the scaling coordinates of the top-left point and the scaling dimensions of the rectangle can be: x2=x1 / n; y2=y1 / n; w2=w1 / n; h2=h1 / n. Where x2, y2, w2, and h2 are the scaling x-coordinate of the top-left point of the target object's rectangle, the scaling y-coordinate of the top-left point of the rectangle, the scaling width of the rectangle, and the scaling height of the rectangle, respectively; n is the ratio of the field of view angles of the main path camera and the auxiliary path camera.

[0077] Optionally, the image dimensions can be set to include the image's width and height.

[0078] Optionally, the mapping location information includes: the mapping coordinates of the top left point of the target object's bounding box and the mapping size information of the bounding box.

[0079] Optionally, the mapped size information of the rectangle includes the mapped width and the mapped height of the rectangle.

[0080] Optionally, the formulas for calculating the top-left point mapping coordinates and the mapping size information of the rectangle are: x3=x2+(fw-fw / n) / 2; y3=y2+(fh-fh / n) / 2; w3=w2; h3=h2. Where x3, y3, w3, and h3 are the top-left point mapping x-coordinate, top-left point mapping y-coordinate, mapping width, and mapping height of the rectangle, respectively; fw and fh are the width and height of the image, respectively.

[0081] Optionally, position compensation is performed on the scaled position information according to the image set size to obtain the mapping position information of the target object in the auxiliary road image, including: using an edge detection algorithm to evaluate the sharpness of the main road image to obtain a sharpness quantification value; determining the position compensation accuracy based on the sharpness quantification value; and performing position compensation on the scaled position information according to the position compensation accuracy and the image set size to obtain the mapping position information of the target object in the auxiliary road image.

[0082] By dynamically adjusting the position compensation accuracy, the stability of auxiliary road image mapping position information calculation under complex lighting conditions is improved.

[0083] Alternatively, the edge detection algorithm can be a Laplacian operator.

[0084] Optionally, the position compensation accuracy is determined based on the sharpness quantification value, including: increasing the position compensation accuracy when the sharpness quantification value is less than a preset sharpness threshold.

[0085] The multi-lens image mapping method provided in this application calculates the mapped position information of the target object in the auxiliary road image by using the position information of the target object in the main road image, along with lens calibration data and ranging information. This process does not require executing a separate detection algorithm on the auxiliary road image; instead, it performs mathematical calculations using the main road image and lens calibration data, thereby reducing computational power consumption and equipment load, extending equipment battery life, and preventing equipment overheating.

[0086] In one embodiment of this application, based on the above embodiment, before step S302, a process of dynamically updating the lens calibration data is further included, as detailed below:

[0087] The lens attitude changes are monitored by built-in sensors to obtain lens attitude information; the lens calibration data is updated based on the lens attitude information.

[0088] Optionally, the built-in sensor can be a gyroscope.

[0089] Optionally, the camera attitude information can be camera tilt or camera movement.

[0090] The multi-lens image mapping method provided in this application ensures the stability of the target object mapping position information in complex environments by dynamically updating lens calibration data and compensating for lens position changes in real time.

[0091] In one embodiment of this application, a multi-lens image mapping method for calibration distance is also provided, the method comprising:

[0092] Step 1: Collect calibration data of the object by using the main road camera and the auxiliary road camera.

[0093] Optionally, calibration data is obtained by acquiring calibration objects using the main road camera and the auxiliary road camera. This includes calibrating two points on the calibration object at a distance from the multi-lens device calibration distance. These two points are the center points of the longitudinal cross-sectional images (main road image and auxiliary road image) acquired by the two cameras at the calibration distance position. The calibration results in the offset of the calibration center points of the two points on the images, including lateral offset and longitudinal offset.

[0094] Figure 5 Example diagrams of images captured by the main path camera and auxiliary path camera provided in embodiments of this application. Figure 5 As shown, optionally, the main road lens can be a main road wide-angle lens, and the auxiliary road lens can be an auxiliary road ultra-wide-angle lens; the red dot is the center point of the longitudinal section image at the calibration distance m1 position acquired by the main road wide-angle lens, and the yellow dot is the center point of the longitudinal section image at the calibration distance m1 position acquired by the auxiliary road ultra-wide-angle lens. In the acquired image, the horizontal offset of the red dot and the yellow dot is dx1, and the vertical offset is dy1.

[0095] Figure 6 Example diagram of the main road image provided in the embodiments of this application; Figure 7 Example diagram of auxiliary road image provided in the embodiments of this application. Figure 6 and Figure 7 As shown, the top-left point of the image is taken as the origin, the X-axis extends horizontally to the right, and the Y-axis extends vertically downwards. From... Figure 6 and Figure 7 The auxiliary path lens has a larger field of view, resulting in a larger scene range in the auxiliary path image, but the objects in the image are relatively smaller. Figure 6 It should be small.

[0096] Optionally, the main road camera is a wide-angle lens; the auxiliary road camera is an ultra-wide-angle lens.

[0097] Optionally, the field of view ratio information can be obtained from the main road image and the auxiliary road image.

[0098] Step 2: Obtain the ratio n of the field of view angles of the main road camera and the auxiliary road camera by using the field of view angle ratio information.

[0099] Step 3: Obtain the location information of the target object in the main road image.

[0100] Optionally, the target object can be Figure 6 and Figure 7 The human face in the middle.

[0101] Step 4: Offset the position information according to the offset of the calibration center point to obtain the offset position information.

[0102] Figure 8 The embodiments provided in this application will Figure 6 Example image after offset.

[0103] Step 5: Scale the offset position information according to the field of view ratio information to obtain the scaled position information.

[0104] Figure 9 The embodiments provided in this application will Figure 8 Example image after scaling.

[0105] Step 6: Obtain the image setting size of the auxiliary path camera.

[0106] Step 7: Perform position compensation on the scaling position information according to the image setting size to obtain the mapping position information of the target object in the auxiliary road image.

[0107] Figure 10 The embodiments provided in this application provide for the Figure 9 Example image after position compensation. (See image below.) Figure 10 As shown, if the image width in the auxiliary lens image setting size is fw, then determine... Figure 9 The width of the image is fw / n; the position compensation accuracy is determined to be (fw-fw / n) / 2.

[0108] Will Figure 10 and Figure 7 A comparison shows that the multi-lens image mapping method provided in this application is effective and accurate, can avoid face detection in auxiliary road images, reduce equipment load, extend battery life, and prevent equipment overheating.

[0109] Figure 11 Example diagrams for face detection in main and auxiliary road images provided in this application. Figure 11 As shown, the detection process specifically includes: acquiring images through the main road wide-angle lens to generate a main road image; acquiring images through the auxiliary road ultra-wide-angle lens to generate an auxiliary road image; performing a face detection algorithm on the main road image to identify and locate the position information of the face region in the image; and mapping the position information using the multi-lens image mapping method provided in this application to obtain the mapped position information of the face region from the auxiliary road perspective. An image processing algorithm is applied to the main road image after face detection; an image processing algorithm is simultaneously applied to the auxiliary road image after mapping processing. The results of processing the main road and auxiliary road images are input into a blurring algorithm, combining the face position, depth information, or viewpoint differences between the two images to achieve the final effect of focusing the target area (such as a face) and blurring the background.

[0110] Figure 12 This is a schematic diagram of the structure of the multi-lens image mapping device provided in the embodiments of this application, as shown below. Figure 12 As shown, the multi-lens image mapping device provided in this embodiment includes: a first acquisition module 1201, a second acquisition module 1202, and a calculation module 1203.

[0111] The first acquisition module 1201 is used to acquire lens calibration data; wherein the lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset is used to indicate the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance is used to indicate the horizontal distance between the calibration object and the main lens or auxiliary lens; the field of view ratio information is used to indicate the ratio relationship between the field of view of the main lens and the auxiliary lens;

[0112] The second acquisition module 1202 is used to acquire the position information and distance information of the target object in the main road image, wherein the main road image is the image of the target object captured by the main road lens, and the distance information is used to indicate the horizontal distance between the target object and the main road lens or the auxiliary road lens.

[0113] The calculation module 1203 is used to calculate the mapping position information of the target object in the auxiliary road image based on the ranging information, lens calibration data and position information, wherein the auxiliary road image is the image of the target object captured by the auxiliary road lens.

[0114] In one possible implementation, the calculation module 1203 is specifically used to: calculate the target center point offset based on lens calibration data and ranging information, wherein the target center point offset is used to indicate the coordinate offset of the image center point of the target object acquired by the auxiliary road lens relative to the image center point of the target object acquired by the main road lens; and calculate the mapping position information of the target object in the auxiliary road image based on the position information, the target center point offset and the field of view ratio information.

[0115] In one possible implementation, the calculation module 1203, when calculating the target center point offset based on lens calibration data and ranging information, is specifically used to: parse the lens calibration data to obtain the calibration center point offset and calibration distance; and calculate the target center point offset using the principle of similar triangles based on the calibration center point offset, calibration distance, and ranging information.

[0116] In one possible implementation, the calculation module 1203, when calculating the mapping position information of the target object in the auxiliary road image based on the position information, the target center point offset, and the field of view ratio information, specifically performs the following: offsetting the position information according to the target center point offset to obtain offset position information; scaling the offset position information according to the field of view ratio information to obtain scaled position information; obtaining the image setting size of the auxiliary road lens; and performing position compensation on the scaled position information according to the image setting size to obtain the mapping position information of the target object in the auxiliary road image.

[0117] In one possible implementation, the multi-lens image mapping device further includes:

[0118] The update module is used to monitor changes in lens attitude through built-in sensors to obtain lens attitude information; and to update the lens calibration data based on the lens attitude information.

[0119] In one possible implementation, the calculation module 1203, when performing position compensation on the scaling position information according to the image set size to obtain the mapping position information of the target object in the auxiliary road image, specifically performs the following: using an edge detection algorithm to evaluate the sharpness of the main road image to obtain a sharpness quantification value; determining the position compensation accuracy based on the sharpness quantification value; and performing position compensation on the scaling position information according to the position compensation accuracy and the image set size to obtain the mapping position information of the target object in the auxiliary road image.

[0120] The multi-lens image mapping device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0121] To implement the above embodiments, this application also provides a chip, which includes at least one processor for executing program instructions to perform the methods as described in any of the above embodiments.

[0122] Figure 13 This is a schematic diagram of the structure of a multi-lens device provided in an embodiment of this application. Optionally, the multi-lens device in this embodiment can be... Figure 2 Multi-lens devices 20. (e.g., ...) Figure 13 As shown, the multi-lens device provided in this embodiment includes at least one processor 1301 and a memory 1302. Optionally, the device further includes a communication component 1303. The processor 1301, memory 1302, and communication component 1303 are connected via a bus 1304.

[0123] In a specific implementation, at least one processor 1301 executes computer execution instructions stored in memory 1302, causing at least one processor 1301 to perform the above-described method.

[0124] The specific implementation process of processor 1301 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0125] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0126] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0127] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0128] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0129] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0130] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0131] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0132] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0135] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0137] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and alterations may be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A multi-lens image mapping method, characterized by, include: Obtain lens calibration data; The lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset indicates the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance indicates the horizontal distance between the calibration object and the main lens or the auxiliary lens; the field of view ratio information indicates the ratio of the field of view of the main lens and the auxiliary lens. The location information and distance measurement information of the target object in the main road image are obtained, wherein the main road image is the image of the target object captured by the main road lens, and the distance measurement information is used to indicate the horizontal distance between the target object and the main road lens or the auxiliary road lens; Based on the ranging information, the lens calibration data, and the position information, the mapping position information of the target object in the auxiliary path image is calculated, wherein the auxiliary path image is an image of the target object captured by the auxiliary path lens.

2. The method of claim 1, wherein, The step of calculating the mapping position information of the target object in the auxiliary road image based on the ranging information, the lens calibration data, and the position information includes: Based on the lens calibration data and the ranging information, the target center point offset is calculated, wherein the target center point offset is used to indicate the coordinate offset of the image center point of the target object captured by the auxiliary lens relative to the image center point of the target object captured by the main lens. Based on the location information, the target center point offset, and the field of view ratio information, the mapping position information of the target object in the auxiliary road image is calculated.

3. The method of claim 2, wherein, The step of calculating the target center point offset based on the lens calibration data and the ranging information includes: The lens calibration data is analyzed to obtain the calibration center point offset and calibration distance; The target center point offset is calculated using the principle of similar triangles based on the calibration center point offset, the calibration distance, and the ranging information.

4. The method of claim 2, wherein, The step of calculating the mapping position information of the target object in the auxiliary path image based on the position information, the target center point offset, and the field of view ratio information includes: The position information is offset based on the target center point offset to obtain the offset position information; The offset position information is scaled proportionally based on the field of view ratio information to obtain scaled position information; Obtain the image setting size of the auxiliary path camera; The scaling position information is compensated according to the image size setting to obtain the mapping position information of the target object in the auxiliary road image.

5. The method according to any one of claims 1 to 4, characterized in that, Before acquiring the location information and ranging information of the target object in the main road image, the method further includes: The lens attitude information is obtained by monitoring changes in lens attitude through built-in sensors. The lens calibration data is updated based on the lens pose information.

6. The method of claim 4, wherein, The step of performing position compensation on the scaling position information according to the image set size to obtain the mapping position information of the target object in the auxiliary road image includes: An edge detection algorithm is used to evaluate the sharpness of the main road image to obtain a sharpness quantification value; The position compensation accuracy is determined based on the aforementioned clear quantification value; The scaling position information is compensated based on the position compensation accuracy and the image setting size to obtain the mapping position information of the target object in the auxiliary road image.

7. A multi-lens image mapping device, comprising: include: The first acquisition module is used to acquire lens calibration data; The lens calibration data includes calibration center point offset, calibration distance, and field of view ratio information; the calibration center point offset indicates the coordinate offset of the image center point of the calibration object acquired by the auxiliary lens relative to the image center point of the calibration object acquired by the main lens; the calibration distance indicates the horizontal distance between the calibration object and the main lens or the auxiliary lens; the field of view ratio information indicates the ratio relationship between the field of view of the main lens and the auxiliary lens. The second acquisition module is used to acquire the position information and distance information of the target object in the main road image, wherein the main road image is the image of the target object captured by the main road lens, and the distance information is used to indicate the horizontal distance between the target object and the main road lens or the auxiliary road lens; The calculation module is used to calculate the mapping position information of the target object in the auxiliary road image based on the ranging information, the lens calibration data and the position information, wherein the auxiliary road image is the image of the target object captured by the auxiliary road lens.

8. A chip, characterized by The chip includes at least one processor, the processor being configured to execute program instructions to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterised in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images