Compound eye imaging ranging system, compound eye imaging ranging method and electronic device

Abstract

The application provides an eye imaging ranging system, an eye imaging ranging method and an electronic device, the eye imaging ranging system comprises: an eye array and an image processing module, the eye array is connected with the image processing module; the eye array comprises at least one long-focus lens and at least one short-focus lens, and the eye array is used for collecting an image signal; the image processing module is used for obtaining a target image based on the image signal, and determining distance information of a target object based on height information of the target object, pixel coordinates of the target object and camera parameter information corresponding to the eye array in the target image. The eye imaging ranging system, the eye imaging ranging method and the electronic device provided by the application can realize real-time imaging, automatically identify a target object, and measure the distance of the target object. The eye array can also meet the demand of long-distance identification and distance measurement, and improve the distance measurement accuracy and distance measurement precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual positioning, in particular to an ommatidium imaging ranging system, an ommatidium imaging ranging method and an electronic device. BACKGROUND

[0002] Bionic ommatidium imaging system is an important technology that is currently developed at home and abroad. People have been researching ommatidium imaging system since the 1920s. Compared with single aperture, the compound eye of insects has the advantages of small volume, large field of view, small distortion, high sensitivity and high dynamic. However, these advantages are at the expense of low resolution, which depends on the number and size of integrated ommatidia. Therefore, the common bionic ommatidium structure has the problems of short shooting distance and low resolution, resulting in low ranging accuracy. SUMMARY

[0003] The present application provides an ommatidium imaging ranging system, an ommatidium imaging ranging method and an electronic device to solve the technical problem of low ranging accuracy of the existing bionic ommatidium system in the prior art.

[0004] The present application provides an ommatidium imaging ranging system, comprising: an ommatidium array and an image processing module, wherein the ommatidium array is connected with the image processing module.

[0005] The ommatidium array comprises at least one long-focus lens and at least one short-focus lens, and the ommatidium array is used for collecting image signals.

[0006] The image processing module is used for obtaining a target image based on the image signals, determining distance information of a target object based on height information of the target object, pixel coordinates of the target object and camera parameter information corresponding to the ommatidium array in the target image.

[0007] In some embodiments, the image processing module comprises an image sensor and an image processing chip, and the image sensor is connected with the image processing chip.

[0008] The image sensor is used for converting light signals containing target information into initial electronic images.

[0009] The image processing chip is used for:

[0010] performing image processing on the initial electronic images to obtain target images conforming to human visual perception;

[0011] performing target recognition on the target images based on a deep learning model to obtain pixel coordinates of a target object in the target images;

[0012] obtaining height information of the target object;

[0013] Determine distance information of the target object based on the pixel coordinates, height information of the target object and the camera parameter information.

[0014] In some embodiments, the image processing module further comprises a network signal transmission module connected with the image processing chip, and the network signal transmission module is configured to transmit the distance information of the target object to a target terminal.

[0015] In some embodiments, the number of the image processing chips is determined based on the number of lenses included in the compound eye array.

[0016] The image sensor is an image information receiving board composed of multiple CMOS chips, and the image processing chip comprises multiple image signal processing core boards.

[0017] The application further provides a compound eye imaging ranging method, comprising:

[0018] Obtain a target image;

[0019] Identify a target object in the target image based on a deep learning model to obtain pixel coordinates of the target object in the target image;

[0020] Obtain height information of the target object;

[0021] Determine distance information of the target object based on the pixel coordinates, height information of the target object and the camera parameter information.

[0022] In some embodiments, the identifying the target object in the target image based on the deep learning model to obtain the pixel coordinates of the target object in the target image comprises:

[0023] Identify the target object in the target image based on the deep learning model to determine candidate box pixel coordinates corresponding to the target object;

[0024] The method further comprises:

[0025] Determine a difference between a maximum value and a minimum value of the longitudinal coordinates in the candidate box pixel coordinates based on the candidate box pixel coordinates.

[0026] In some embodiments, the distance information of the target object is:

[0027]

[0028] Wherein, Z is the distance information of the target object, D is the height information of the target object, fy is the camera parameter information, and n is the difference between the maximum value and the minimum value of the longitudinal coordinates in the candidate box pixel coordinates.

[0029] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the compound eye imaging ranging method according to any one of the above when executing the program.

[0030] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the compound eye imaging ranging method according to any one of the above.

[0031] The application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the compound eye imaging ranging method according to any one of the above.

[0032] The compound eye imaging ranging system, the compound eye imaging ranging method and the electronic device provided by the application can realize ranging on a target object by identifying a target image, obtaining pixel coordinates of the target object, and constructing a ranging algorithm according to the pixel coordinates, height information of the target object and camera parameter information, thereby improving ranging accuracy and ranging precision. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0034] Figure 1 is a structural schematic diagram of the compound eye imaging ranging system provided by the application;

[0035] Figure 2 is a structural schematic diagram of the compound eye array in the compound eye imaging ranging system provided by the application;

[0036] Figure 3 is a structural schematic diagram of the image processing module in the compound eye imaging ranging system provided by the application;

[0037] Figure 4 is a structural schematic diagram of the compound eye imaging ranging system provided by the application;

[0038] Figure 5 is a structural schematic diagram of the internal assembly structure of the compound eye imaging ranging system provided by the application;

[0039] Figure 6 is a flow schematic diagram of the compound eye imaging ranging method provided by the application;

[0040] Figure 7is a ranging principle schematic diagram of the compound eye imaging ranging method provided by the present application;

[0041] Figure 8 is a structural schematic diagram of the ranging device provided by the present application;

[0042] Figure 9 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0043] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0044] The compound eye imaging ranging system, the compound eye imaging ranging method and the electronic device provided by the present application will be described below with reference to the drawings. Figures 1-9 The compound eye imaging ranging system, the compound eye imaging ranging method and the electronic device provided by the present application will be described below with reference to the drawings.

[0045] Figure 1 is a structural schematic diagram of the compound eye imaging ranging system provided by the present application. With reference to Figure 1 The compound eye imaging ranging method provided by the present application comprises a compound eye array 110 and an image processing module 120, and the compound eye array 110 is connected with the image processing module 120.

[0046] The compound eye array 110 comprises at least one long-focus lens and at least one short-focus lens, and the compound eye array is used for collecting image signals.

[0047] The compound eye array 110 is composed of multiple lenses. The long-focus lens has a small field of view and a long shooting distance, can obtain image signals with clear details, and is used for identifying and ranging a target object. The short-focus lens has a large field of view and a short shooting distance, and is used for expanding the field of view.

[0048] The parameters of the focal length size of the compound eye array 110 and the installation position can be determined based on the distance of ranging and the image plane size and position of the CMOS chip.

[0049] In actual execution, in order to reduce the cost of the device, only one long-focus lens and one short-focus lens can be arranged.

[0050] In the embodiment, Figure 2 is a structural schematic diagram of the compound eye array in the compound eye imaging ranging system provided by the present application, as Figure 2As shown, the compound eye array can consist of a central telephoto lens and four surrounding short-focus lenses. There can be six or eight short-focus lenses, etc., so that the short-focus lenses can be symmetrically arranged around the telephoto lens. There can also be five or seven short-focus lenses, etc., without specific limitations here.

[0051] The image processing module 120 is used to obtain a target image based on the image signal, and to determine the distance information of the target object based on the height information of the target object in the target image, the pixel coordinates of the target object, and the camera parameter information corresponding to the compound eye array.

[0052] The image processing module 120 is equipped with a target recognition algorithm and a target ranging algorithm that combines prior features of the target.

[0053] Target recognition algorithms are used to identify target objects in target images.

[0054] The target ranging algorithm calculates the distance to the target object based on the height information of the target object in the target image, the pixel coordinates of the target object, and the camera parameter information corresponding to the compound eye array.

[0055] Figure 3 This is a schematic diagram of the image processing module structure in the compound eye imaging ranging system provided by the present invention, as shown below. Figure 3 As shown, the image processing module 120 may include a housing 320.

[0056] The housing 320 may include a compound eye mount and a focusing bracket 310. The housing 320 is mainly used to fix various processing plates and lenses.

[0057] The compound eye mounting and focusing bracket 310 is used to fix the compound eye array and adjust the focusing distance of the compound eye array.

[0058] The housing 320 also includes a heat dissipation structure to prevent the compound eye imaging ranging system from overheating.

[0059] Figure 4 This is a schematic diagram of the overall structure of the compound eye imaging ranging system provided by the present invention, as shown below. Figure 4 As shown, Figure 2 The compound eye array in the middle can be installed by threading onto Figure 3 The compound eye is mounted on the focusing bracket 310.

[0060] The compound eye imaging ranging system provided by this invention can achieve long-distance ranging through a compound eye array, while improving ranging accuracy and precision. It solves the contradiction between existing compound eye imaging ranging systems, where a large field of view is convenient for finding targets but has low ranging accuracy, and a small field of view has high ranging accuracy but is not suitable for finding targets.

[0061] In some embodiments, the image processing module 120 comprises an image sensor and an image processing chip, the image sensor being connected to the image processing chip;

[0062] The image sensor is used to convert the light signal containing target information into an initial electronic image;

[0063] The image processing chip is used to:

[0064] perform image processing on the initial electronic image to obtain a target image;

[0065] perform target recognition on the target image based on a deep learning model to obtain pixel coordinates of a target object in the target image;

[0066] obtain height information of the target object;

[0067] determine distance information of the target object based on the pixel coordinates, the height information of the target object, and camera parameter information.

[0068] In actual implementation, the image processing module 120 can comprise an image sensor and an image processing chip.

[0069] In some embodiments, the image sensor is an image information receiving board composed of multiple CMOS chips, and the image processing chip comprises multiple image signal processing core boards.

[0070] The image sensor in the compound eye imaging ranging system can be an image information receiving board composed of multiple CMOS chips.

[0071] The image sensor is used to convert the light signal on its light receiving surface into a usable electrical signal, i.e., the image signal is the light signal, and the initial electronic image is the usable electrical signal. In actual implementation, the CMOS chip converts the light signal into a digital signal to generate a RAW image.

[0072] The RAW image is the original data of the digital signal converted by the image sensor from the captured light source signal. The RAW image is an initial electronic image that does not conform to human visual perception.

[0073] The image processing chip can be an image signal processing core board composed of 38mm*38mm rv1126 core boards.

[0074] The core board has an image signal processing (ISP) chip that can convert the RAW image into an image that conforms to the normal perception of human visual perception, i.e., a target image.

[0075] The image signal processing core board also has a video encoding output function. It can input image signals to the video encoder (Vedio Encode, VENC) for encoding through the video input (VedioInput, VI) channel, and output video streams based on the Real Time Streaming Protocol (RTSP) network.

[0076] like Figure 5 As shown, this embodiment is illustrated using a compound eye array 110 comprising five lenses (one telephoto lens and four short-focus lenses).

[0077] Understandably, the five-lens system can acquire five image signals. The image processing module 120 may at this time include one CMOS chip 510 and three RV1126 image signal processing core boards 520.

[0078] Three RV1126 core boards 520 process the initial electronic image data sent by the CMOS chip 510. Two of the RV1126 core boards process four channels of initial electronic image data acquired by the short-focus lens, with each board processing two channels. The remaining RV1126 core board processes one channel of initial electronic image data acquired by the intermediate telephoto lens and performs target recognition and ranging functions.

[0079] In actual operation, external light shines on the CMOS chip 510 through one telephoto lens and four short-focus lenses. The CMOS chip 510 converts the light signal into a digital signal to generate a RAW image.

[0080] The CMOS chip 510 sends RAW image data to three core boards, which can convert the RAW image into an image that conforms to the normal visual perception of the human eye.

[0081] The middle telephoto lens requires recognition and ranging, so a deep learning model and target ranging algorithm compatible with the 520 core board are added between the VI channel and VENC. The four short-focus lenses do not require recognition and ranging; the image signals can be directly input to the VENC through the VI channel for encoding, and the video stream is output based on the RTSP network.

[0082] The CMOS chip 510 acquires the initial electronic image; that is, the CMOS chip 510 can acquire video signals or capture single-frame images. Image processing and encoding into a video stream for output are completed on the core board and can be controlled by the core board.

[0083] In some embodiments, the image processing module further includes a network signal transmission module connected to the image processing chip, which is used to transmit distance information of the target object to the target terminal.

[0084] The image processing module 120 further comprises a network signal transmission module.

[0085] The network connection of the image processing chip can be connected to form a network port for network transmission.

[0086] Figure 5 is a schematic diagram of the internal assembly structure of the compound eye imaging ranging system provided by the application, as shown in Figure 5 Take the compound eye array 110 comprising five lenses (one long-focus lens and four short-focus lenses) as an example to illustrate the embodiment.

[0087] The image processing module 120 can comprise one CMOS chip 510, three rv1126 image signal processing core boards 520, one network signal transmission board 530, and one connecting bottom plate 540.

[0088] The above-mentioned board members can be installed according to the assembly structure shown in Figure 5 The assembly structure from left to right is one CMOS chip 510, three rv1126 image signal processing core boards 520, and one network signal transmission board 530. One connecting bottom plate 540 is used to fix the three rv1126 image signal processing core boards 520.

[0089] The deep learning model in the embodiment of the application can be a model for target detection, for example, can be: a region-based CNN (R-CNN) model, a YOLO model, an Inception model, a single shot multi box detector (SSD) model, etc.

[0090] After the deep learning model is trained, it can be converted into a general format of onnx, and then converted into an rknn format suitable for the core board through a tool. Then a post-processing program code of the deep learning model is written, and the compiled code is sent to the board end to run to realize the functions of target recognition and ranging, which can be used to obtain pixel coordinates.

[0091] In actual execution, the compound eye array 110 sends the obtained initial electronic image to the image processing module 120 for image processing, that is, the target image is input to the deep learning model in the image processing module 120, and an image with a target detection frame output by the deep learning model is obtained, the target detection frame is a candidate frame, and then the pixel coordinates of the target object can be determined.

[0092] The compound eye array is calibrated by a checkerboard target to obtain camera parameter information.

[0093] The height information of the target object in the target image is determined by measuring the target object. For example, the height information of a 1.8m person, a 1.5m high car, etc. is measured and obtained.

[0094] According to the pixel coordinates, the height information of the target object, and the camera parameter information obtained in the above steps, the distance information of the target object can be calculated according to the equation relationship in the above embodiment, that is, the ranging of the target object can be completed, and details are not repeated here.

[0095] The compound eye imaging ranging system provided by the application can realize identification of the target object through the deep learning model, and can realize ranging of the target object according to the ranging algorithm composed of the pixel coordinates, the height information of the target object, and the camera parameter information, thereby improving the ranging accuracy and ranging precision.

[0096] In some embodiments, the number of image processing chips is determined based on the number of lenses included in the compound eye array.

[0097] In actual implementation, 1 long-focus lens needs 1 image processing chip, and 2 short-focus lenses need 1 image processing chip.

[0098] Therefore, in the case that the compound eye array includes 1 long-focus lens and a plurality of short-focus lenses, the scheme of 1 long-focus lens and 4 short-focus lenses needs 3 image processing chips, the scheme of 1 long-focus lens and 6 short-focus lenses needs 4 image processing chips, and the scheme of 1 long-focus lens and 8 short-focus lenses needs 5 image processing chips.

[0099] The compound eye imaging ranging system provided by the application can flexibly set the image processing chip through the number of lenses, so that the ranging precision can be determined according to the requirements, and the ranging flexibility is improved.

[0100] Figure 6 is a flowchart of the compound eye imaging ranging method provided by the application. Referring to Figure 6 The compound eye imaging ranging method provided by the application includes steps 610, 620, 630, and 640.

[0101] The execution subject of the compound eye imaging distance measurement method provided by the application can be an electronic device, a component in the electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the application is not limited in this regard.

[0102] The technical solution of the application will be described in detail below with the example of an electronic device executing the compound eye imaging distance measurement method provided by the application.

[0103] Step 610: Obtain a target image.

[0104] In actual execution, the target image can be a static image, for example, a photo, or a dynamic image, for example, a video, and the specific implementation can be determined according to actual use requirements, and the embodiments of the application are not limited in this regard.

[0105] The electronic device can obtain the target image from a local storage space, from another electronic device, from a network, by a photographing module, or by any other possible way, and the specific implementation can be determined according to actual use requirements, and the embodiments of the application are not limited in this regard.

[0106] Step 620: Perform target recognition on the target image based on a deep learning model to obtain the pixel coordinates of the target object in the target image.

[0107] The deep learning model in the embodiments of the application can be a model for target detection, for example, a Region-based CNN (R-CNN) model, a YOLO model, an Inception model, or a Single Shot MultiBox Detector (SSD) model.

[0108] The target image is input into the deep learning model to obtain an image with a target detection frame output by the deep learning model, and the target detection frame is a candidate frame. The image content selected by the candidate frame is a target object. The target object can be a person, a vehicle, or an animal or plant, and is not specifically limited here.

[0109] Based on the image with the target detection frame output by the deep learning model, the pixel coordinates of the candidate frame can be determined.

[0110] The pixel coordinates of the candidate frame can be pixel coordinates of vertices of the candidate frame or pixel coordinates corresponding to points on the side length of the candidate frame.

[0111] In some embodiments, step 620 can specifically include:

[0112] Based on the target recognition of the target image by the deep learning model, the pixel coordinates of the candidate frame corresponding to the target object are determined.

[0113] The compound eye imaging distance measurement method further includes determining the difference between the maximum longitudinal coordinate and the minimum longitudinal coordinate in the pixel coordinates of the candidate frame based on the pixel coordinates of the candidate frame.

[0114] After the pixel coordinates of the candidate frame are determined, the candidate frame is a rectangle, and the four vertex coordinates can include: a top-left vertex coordinate, a bottom-right vertex coordinate, a bottom-left vertex coordinate, and a top-right vertex coordinate.

[0115] In actual execution, the top-left vertex coordinate can be set as (x1, y1), the bottom-right vertex coordinate can be set as (x2, y2), the bottom-left vertex coordinate can be set as (x1, y2), and the top-right vertex coordinate can be set as (x2, y1).

[0116] Further, the difference between the maximum longitudinal coordinate and the minimum longitudinal coordinate in the pixel coordinates of the candidate frame can be determined as y2-y1.

[0117] The compound eye imaging distance measurement method provided by the application can detect a target object through a deep learning model for a target image, obtain pixel coordinates of the target object, and improve the accuracy and detection efficiency of target recognition.

[0118] Step 630: obtaining height information of the target object.

[0119] The chessboard target is calibrated for the shooting module in the electronic device to obtain camera parameter information. The camera parameter information can be camera intrinsic parameters. The commonly used camera intrinsic parameters can include fx, fy, (u0, v0).

[0120] fx=f / dx, fy=f / dy, fx and fy are called normalized focal length on x-axis and y-axis respectively, f is the focal length of the lens, the unit is generally mm, dx and dy are the pixel size; wherein, fx=fy.

[0121] u0 and v0 respectively represent the horizontal pixel number and the vertical pixel number between the center pixel coordinates of the image and the image origin pixel coordinates.

[0122] In actual execution, after the focal length is obtained, the height information of the target object can be determined according to the proportion of the target object in the target image and the different focal length magnification coefficients.

[0123] Or directly through the measurement of the target object, the height information of the target object is obtained.

[0124] Step 640, based on the pixel coordinates, the height information of the target object and the camera parameter information, the distance information of the target object is determined.

[0125] According to the pixel coordinates, the height information of the target object and the camera parameter information obtained in the above steps, the distance information of the target object can be calculated, that is, the distance measurement of the target object can be completed.

[0126] The compound eye imaging distance measurement method provided by the application can realize the identification of the target object through the deep learning model, and can realize the distance measurement of the target object according to the distance measurement algorithm composed of the pixel coordinates, the height information of the target object and the camera parameter information, thereby improving the distance measurement accuracy and distance measurement precision.

[0127] In some embodiments, the distance information of the target object is:

[0128]

[0129] Wherein, Z is the distance information of the target object, D is the height information of the target object, fy is the camera parameter information, and n is the difference between the maximum vertical coordinate and the minimum vertical coordinate in the candidate box pixel coordinates.

[0130] Figure 7 The distance measurement principle diagram of the compound eye imaging distance measurement method provided by the application is shown in FIG. 1. Figure 7 As shown in the figure, the following proportional relationship can be obtained according to the trigonometric function relationship:

[0131]

[0132] And fy=fx=f / dx (f is the focal length of the lens, and dx is the pixel size), then the distance information Z of the target object is:

[0133]

[0134] D is height information of the target object, fy is camera parameter information, and n is a difference between a maximum longitudinal coordinate and a minimum longitudinal coordinate in pixel coordinates of the candidate frame.

[0135] In some embodiments, the lens is calibrated by a chessboard target, and the camera parameter information is obtained, and the target object is a target person, and the height of the target person is measured to obtain the height information of the target person.

[0136] Suppose that the height of the target person at a distance of 80 meters is 1.8 meters, fy = f / dy = 8700, coordinate y2 = 429, and coordinate y1 = 234, then the distance information Z of the target object is:

[0137]

[0138] The real-time distance of the target object can be measured according to the compound eye imaging distance measurement method, and the measurement result can be compared with the true value and the error accuracy can be analyzed.

[0139] In some embodiments, the measurement result is shown in the following table:

[0140] True value 30m 40m 50m 60m 70m 80m 90m 100m Measured value 29.27m 38.86m 49.25m 60.46m 68.68m 80.31m 88.5m 101.1m Error 2.4％ 2.85％ 1.5％ 0.77％ 1.9％ 0.39％ 1.67％ 1.1％

[0141] As shown in the above table, the compound eye imaging distance measurement method provided in the embodiments has a measurement accuracy within 3% for a target person at a distance of 30-100 meters, and has good measurement accuracy.

[0142] The compound eye imaging distance measurement method provided in the embodiments can determine the distance information of the target object in real time, and improve the distance measurement efficiency.

[0143] The distance measurement device provided in the embodiments is described below, and the distance measurement device described below can be correspondingly referred to the compound eye imaging distance measurement method described above.

[0144] Figure 8 is a structural schematic diagram of the distance measurement device provided in the embodiments. Referring to Figure 8 The distance measurement device provided in the embodiments includes an acquisition module 810, a first determination module 820, a second determination module 830, and a third determination module 840.

[0145] The acquisition module 810 is configured to acquire a target image.

[0146] The first determination module 820 is configured to perform target recognition on the target image based on a deep learning model to obtain pixel coordinates of a target object in the target image.

[0147] The second determination module 830 is configured to acquire height information of the target object.

[0148] The third determining module 840 is configured to determine distance information of the target object based on the pixel coordinates, height information of the target object, and the camera parameter information.

[0149] The ranging device provided by the application can realize ranging of the target object and improve ranging accuracy and ranging precision by identifying the target image to obtain pixel coordinates of the target object and using a ranging algorithm composed of the pixel coordinates, height information of the target object, and camera parameter information.

[0150] In some embodiments, the first determining module 820 is further configured to:

[0151] The target object is identified based on the deep learning model to determine candidate box pixel coordinates corresponding to the target object;

[0152] The device further includes:

[0153] The third determining module is configured to determine a difference between a maximum value and a minimum value of the longitudinal coordinates in the candidate box pixel coordinates based on the candidate box pixel coordinates.

[0154] In some embodiments, the distance information of the target object is:

[0155]

[0156] wherein Z is the distance information of the target object, D is the height information of the target object, fy is the camera parameter information, and n is the difference between the maximum value and the minimum value of the longitudinal coordinates in the candidate box pixel coordinates.

[0157] Figure 9 An example of a schematic diagram of a physical structure of an electronic device is shown in FIG. 10. Figure 9 As shown in FIG. 10, the electronic device can include a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 can communicate with each other through the communications bus 940. The processor 910 can invoke a logical instruction in the memory 930 to execute a compound eye imaging ranging method, which includes:

[0158] obtaining a target image;

[0159] identifying the target object in the target image based on a deep learning model to obtain pixel coordinates of the target object in the target image;

[0160] obtaining height information of the target object;

[0161] determine distance information of the target object based on the pixel coordinates, the height information of the target object, and the camera parameter information.

[0162] In addition, the logic instructions in the memory 930 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0163] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the compound eye imaging ranging method provided by the above-mentioned methods, which comprises:

[0164] obtaining a target image;

[0165] performing target recognition on the target image based on a deep learning model to obtain pixel coordinates of a target object in the target image;

[0166] obtaining height information of the target object;

[0167] determining distance information of the target object based on the pixel coordinates, the height information of the target object, and the camera parameter information.

[0168] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the compound eye imaging ranging method provided by the above-mentioned methods, which comprises:

[0169] obtaining a target image;

[0170] performing target recognition on the target image based on a deep learning model to obtain pixel coordinates of a target object in the target image;

[0171] obtaining height information of the target object;

[0172] Based on the pixel coordinates, height information of the target object and the camera parameter information, distance information of the target object is determined.

[0173] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0174] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0175] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A compound eye imaging ranging system, characterized in that, include: A compound eye array and an image processing module, wherein the compound eye array is connected to the image processing module; The compound eye array includes at least one telephoto lens and at least one short focal length lens. The short focal length lens is used to locate the target, and the telephoto lens is used for identification and ranging. The compound eye array is used to collect light signals containing target information as image signals. The focal length parameters and installation position of the compound eye array are determined based on the distance of the ranging and the image plane size and position of the CMOS chip. The image processing module includes an image sensor and an image processing chip, and the image sensor is connected to the image processing chip. The image sensor is an image information receiving board composed of multiple CMOS chips, used to convert light signals containing target information into an initial electronic image; The image processing chip includes multiple image information processing core boards, the number of which is determined based on the number of lenses included in the compound eye array. Each telephoto lens corresponds to an independent chip, and its computing power is preferentially allocated to target recognition and ranging. Each short-focus lens is allocated one chip for every two channels, and is used for: The initial electronic image is processed to obtain a target image that conforms to human visual perception; The target image is identified based on a deep learning model that uses an adapted hardware format to obtain the pixel coordinates of candidate bounding boxes of the target object in the target image, wherein the adapted hardware format is suitable for the image processing chip. Obtain the height information of the target object; The target image that has been identified is encoded based on the video encoder; The image processing module determines the distance information of the target object based on the pixel coordinates of the candidate box, the height information of the target object, and the camera parameter information; The image processing module also includes a network signal transmission module, which is connected to the image processing chip. The network signal transmission module is used to transmit the distance information of the target object to the target terminal and output the network video stream in real time based on the real-time streaming media transmission protocol.

2. A compound eye imaging ranging method, characterized in that, The compound eye imaging ranging system according to claim 1 includes: The light signal containing target information is collected by a compound eye array as an image signal. The light signal containing target information is converted into an initial electronic image by an image sensor. The initial electronic image is processed by an image processing chip to obtain a target image that conforms to human vision. The image processing chip performs target recognition on the target image based on a deep learning model using an adapted hardware format, and obtains the pixel coordinates of the candidate bounding box of the target object in the target image. The height information of the target object is obtained through the image processing chip. The target image that has been identified is encoded based on the video encoder; The image processing chip determines the distance information of the target object based on the candidate box pixel coordinates, the height information of the target object, and the camera parameter information.

3. The compound eye imaging ranging method according to claim 2, characterized in that, The step of performing target recognition on the target image using a deep learning model adapted to the hardware format, and obtaining the candidate bounding box pixel coordinates of the target object in the target image, includes: Based on the deep learning model, target recognition is performed on the target image to determine the pixel coordinates of the candidate bounding box corresponding to the target object; The method further includes: Based on the candidate box pixel coordinates, determine the difference between the maximum and minimum ordinate values ​​in the candidate box pixel coordinates.

4. The compound eye imaging ranging method according to claim 3, characterized in that, The distance information of the target object is: Where Z represents the distance information of the target object, D represents the height information of the target object, fy represents the camera parameter information, and n represents the difference between the maximum and minimum values ​​of the ordinate in the candidate box pixel coordinates.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the compound eye imaging ranging method as described in any one of claims 2-4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the compound eye imaging ranging method as described in any one of claims 2-4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the compound eye imaging ranging method as described in any one of claims 2-4.

Images