Distance measurement method and device based on binocular camera, storage medium and electronic equipment

Abstract

The invention discloses a distance measuring method and device based on a binocular camera, a storage medium and electronic equipment. The method comprises the following steps: acquiring an original image based on a binocular camera, and preprocessing the original image; generating a disparity map based on the preprocessed original image, and converting the disparity map into a depth map; inputting the depth map into a neural network model based on deep learning for optimization to obtain an optimized depth map; inputting the original image into a target detection model to obtain the category and bounding box information of a target; and obtaining the position information of the target based on the bounding box information and the optimized depth map. According to the invention, the speed and precision of distance measurement can be effectively improved, and strong support can be provided for object positioning and tracking in a complex environment.

Description

Technical Field

[0001] The present application relates to the field of computer vision and deep learning technology, and in particular to a ranging method, device, storage medium and electronic device based on a binocular camera. Background Art

[0002] With the annual increase in my country's power load, the quality requirements for power grids and related power system facilities have been significantly improved. In many cases, live working needs to ensure the normal operation of facilities. In terms of standardization, a large number of live working procedures for ensuring safe operations have been specified, such as different distance requirements for different voltage levels, such as the minimum grounding distance, the minimum safe distance for ground potential, the minimum distance between adjacent phases, and the combined gap distance. In terms of digitalization, in recent years, the State Grid has used digital twins as its goal and used new technical means to obtain a large amount of high-precision power infrastructure information, providing effective data support for the safe and accurate control of live working.

[0003] Existing distance measurement methods are based on monocular cameras. Monocular vision-based methods are difficult to directly obtain accurate depth information. Although sensors such as lidar can provide precise distance data, they are expensive and large in size, limiting their applicability in certain scenarios. Summary of the invention

[0004] The embodiments of the present application provide a binocular camera-based distance measurement method, device, storage medium and electronic device, which can effectively improve the speed and accuracy of distance measurement, and can provide strong support for object positioning and tracking in complex environments.

[0005] The present application provides a distance measurement method based on a binocular camera, including: Acquire an original image based on a binocular camera, and preprocess the original image; Generate a disparity map based on the preprocessed original image, and convert the disparity map into a depth map; Inputting the depth map into a deep learning-based neural network model for optimization to obtain an optimized depth map; Inputting the original image into the target detection model to obtain the category and bounding box information of the target; Based on the bounding box information and the optimized depth map, the location information of the target is obtained.

[0006] Furthermore, in the above-mentioned ranging method based on a binocular camera, the original image includes a left image and a right image, and the disparity map is generated based on the preprocessed original image, including: For each pixel point, calculating the matching cost of the pixel point in the left image and the right image; Dynamically program pixels along multiple directions to obtain aggregate costs; The disparity value of the pixel corresponding to the minimum aggregation cost is taken as the optimal disparity value, and a disparity map is constructed based on the optimal disparity value of each pixel.

[0007] Furthermore, in the above-mentioned distance measurement method based on binocular camera, the dynamic programming is calculated by the following formula:

[0008] in, represents the current pixel position, represents parallax, represents the step length along a certain direction, represents the original cost, represents the cost after aggregation, and For penalty items.

[0009] Furthermore, in the above-mentioned distance measurement method based on a binocular camera, converting the disparity map into a depth map comprises: The disparity value in the disparity map is converted into a depth value in the depth map by a first formula, and the first formula is:

[0010] Among them, Z is the depth value, B is the baseline length, f is the focal length, is the parallax value.

[0011] Furthermore, in the above-mentioned distance measurement method based on binocular camera, the neural network model based on deep learning includes an encoder and a decoder; The encoder includes multiple convolutional layers, each of which is connected to a ReLU activation function layer and a maximum pooling layer. The output of each layer of the encoder is:

[0012] in, It is The feature map of the layer, Conv represents the convolution operation, and ReLU is the activation function; The decoder is used to restore the original resolution of the input feature map and fuse multi-scale feature information. The fused feature map is expressed as:

[0013] in, is a learnable weight parameter.

[0014] Furthermore, the above-mentioned distance measurement method based on binocular camera, wherein the method also includes: A self-attention mechanism is introduced into the neural network model based on deep learning, and the attention weight is expressed as:

[0015] in, is the attention weight, Q is the query, K is the key, T is the transpose of the matrix, is the dimension of the key; Final attention output for:

[0016] Among them, V is the value matrix.

[0017] Furthermore, in the above-mentioned distance measurement method based on a binocular camera, the original image is preprocessed, including: The original image is subjected to denoising and edge preservation processing by bilateral filtering, and the formula corresponding to the bilateral filtering is:

[0018] in, Pixel The strength value of is the spatial Gaussian weight, based on the spatial distance of pixels, is the pixel intensity difference weight; is a normalization factor used to ensure that the filtered value is within a reasonable range.

[0019] The embodiment of the present application also provides a distance measurement device based on a binocular camera, comprising: An acquisition and preprocessing module, used to acquire original images based on a binocular camera and preprocess the original images; A depth map generation module, used to generate a disparity map based on the preprocessed original image, and convert the disparity map into a depth map; An optimization module, used for inputting the depth map into a neural network model based on deep learning for optimization, to obtain an optimized depth map; An object detection module is used to input the original image into an object detection model to obtain the category and bounding box information of the object; A positioning module is used to obtain the position information of the target based on the bounding box information and the optimized depth map.

[0020] An embodiment of the present application also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are suitable for being loaded by a processor to execute any of the above-mentioned ranging methods based on a binocular camera.

[0021] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to perform the steps in any of the above-mentioned ranging methods based on a binocular camera.

[0022] The present application provides a binocular camera-based ranging method, device, storage medium and electronic device. The present application captures images at different viewing angles through a binocular camera, generates depth maps through images at different viewing angles, and performs target recognition through optimized depth maps and original images. This method can not only effectively improve the speed and accuracy of distance measurement, but also provide strong support for object positioning and tracking in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0024] Figure 1 A flowchart of a binocular camera-based ranging method provided in an embodiment of the present application.

[0025] Figure 2 Another flow chart of the ranging method based on a binocular camera provided in an embodiment of the present application.

[0026] Figure 3 A schematic diagram of the bilateral filtering principle provided in an embodiment of the present application.

[0027] Figure 4 A schematic diagram of the structure of a binocular camera-based distance measuring device provided in an embodiment of the present application.

[0028] Figure 5 A flowchart of obtaining an original image through a binocular camera provided in an embodiment of the present application.

[0029] Figure 6 A user interaction flow chart provided for an embodiment of the present application.

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0032] The embodiments of the present application provide a binocular camera-based distance measurement method, device, storage medium, and electronic device. The binocular camera-based distance measurement device provided in the embodiments of the present application can be integrated in an electronic device, which can be a terminal, a server, and the like, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro-processing box, or other devices.

[0033] See also Figure 1 and Figure 2 , Figure 1 A flow chart of a binocular camera-based distance measurement method provided in an embodiment of the present application. Figure 2 Another flow chart of a binocular camera-based distance measurement method provided in an embodiment of the present application, which is applied to an electronic device, and the binocular camera-based distance measurement method includes the following steps: S1, obtains the original image based on the binocular camera and preprocesses the original image.

[0034] The binocular camera achieves synchronous operation through the internal control module, ensuring that the left image and the right image are captured at the same time to avoid data mismatch caused by time difference. The collected original image is temporarily stored through the data storage module inside the camera to provide reliable data input for subsequent processing.

[0035] Figure 3 The bilateral filtering principle diagram provided in the embodiment of the present application is as follows: Figure 3 As shown, Figure 3 It shows how to smooth an image while preserving edge details by combining spatial information and pixel intensity information. The input image p represents the raw data to be processed, which may usually contain noise or high-frequency texture, while the guidance image I provides the basis for the filter to calculate the weights. The guidance image can be the input image itself or another related image to enhance the representation of edge and texture information. In the filtering process, the spatial kernel Weights are assigned based on the geometric distance between pixels. The closer to the target pixel, the higher the weight, thus achieving a smoothing effect within the spatial range. The similarity is evaluated based on the difference in pixel intensity, and similar pixels are given higher weights to protect the edges and textures in the image. Bilateral filtering is formed by combining these two kernels to form a bilateral kernel. , the neighborhood of each pixel is weighted averaged to generate the final filtered output q. The output image can effectively remove noise and retain the clarity of edge information, making it more robust and accurate in subsequent image processing tasks (stereo matching and depth estimation).

[0036] The preprocessing of the original image includes: denoising and edge preservation of the original image (left image and right image) by bilateral filtering. The formula corresponding to the bilateral filtering is:

[0037] in, Pixel The strength value of is the spatial Gaussian weight, based on the spatial distance of pixels, is the pixel intensity difference weight; is a normalization factor used to ensure that the filtered value is within a reasonable range.

[0038] Furthermore, in order to highlight the details of the scene, histogram equalization or adaptive contrast enhancement algorithm can be used to optimize the brightness and contrast of the image, and the image can be geometrically corrected by combining the calibration parameters (intrinsic parameter matrix and distortion coefficient) of the binocular camera to provide high-quality input images for stereo matching.

[0039] S2, generating a disparity map based on the preprocessed original image, and converting the disparity map into a depth map.

[0040] In one embodiment, a semi-global block matching (SGBM) algorithm is used to calculate the disparity map, and step S2 includes the following steps: S21, for each pixel point, calculating the matching cost of the pixel point in the left image and the right image.

[0041] Specifically, for each pixel position, the difference between its corresponding position in the left image and the corresponding position in the right image is calculated. Commonly used cost functions include absolute difference (AD), squared difference (SD), or normalized cross-correlation (NCC).

[0042] Calculate the matching cost using the AD method:

[0043] in, is the pixel value of the left image, is the pixel value of the corresponding point in the left image on the right image after it is offset by d in the x direction.

[0044] pass Method to calculate matching cost:

[0045] Calculate the matching cost using the NCC method:

[0046] in, is the pixel value of the left image, is the pixel value of the corresponding point on the left image after it is offset by d in the x direction. To match the window, is the mean value of the pixels of the left view in the original window, The right view is the mean value of the pixels within the matching window.

[0047] S22, dynamically program the pixels along multiple directions to obtain the aggregation cost.

[0048] Specifically, by performing dynamic programming along multiple directions (usually 8 directions), the local cost is expanded to a larger area, thereby reducing the impact of noise and enhancing the consistency of disparity. This step is the core of SGBM and can effectively handle occlusion problems.

[0049] Dynamic programming is calculated by the following formula:

[0050] in, represents the current pixel position, represents parallax, represents the step length along a certain direction, represents the original cost, represents the cost after aggregation, and For penalty items.

[0051] Cost aggregation is a step that is performed after the initial calculation of the matching cost between each pixel and the possible disparity values. In SGBM, this step is to reduce the impact of noise and improve the accuracy of disparity estimation by considering contextual information in a larger range. For each pixel in the image, the matching cost under different disparity assumptions with the corresponding position in the right image is calculated. This matching cost can be based on the sum of absolute differences (SAD). Dynamic programming on a one-dimensional path is then performed along multiple directions (usually 8 or 16 directions). For each direction, all pixels are traversed inward from the edge of the image, and the cost of each pixel under a specific disparity hypothesis is accumulated. In this process, smoothness constraints (such as P1 and P2 penalties) are introduced to encourage neighboring pixels to have similar disparity values ​​unless there is a clear edge indicating that the disparity should change. The final aggregated cost for each pixel is the sum of the path costs in all directions. The purpose of this is to combine local and global information to obtain a more accurate disparity estimate.

[0052] S23, taking the disparity value of the pixel corresponding to the minimum aggregation cost as the optimal disparity value, and constructing a disparity map based on the optimal disparity value of each pixel.

[0053] The best disparity value for each pixel is selected, usually the one that minimizes the aggregation cost. In addition, unreasonable disparity values ​​can be further eliminated through consistency checks, such as left-right consistency checks.

[0054] Furthermore, after constructing the disparity map, the quality of the preliminary depth map needs to be improved. The following methods can be used: Median filtering: Use a 3x3 or 5x5 median filter to remove isolated noise points, keeping the edges clear while making the disparity map smoother.

[0055] Bilateral filtering: Smoothes the disparity map while preserving edge information by taking into account both spatial proximity and intensity similarity.

[0056] Hole filling: For hole areas caused by occlusion and other reasons, interpolation methods can be used to fill them, such as the average value based on the neighborhood or the propagation based on known disparity.

[0057] Disparity refinement: Improve the accuracy of disparity estimation through sub-pixel precision disparity refinement techniques such as quadratic fitting or cubic polynomial fitting.

[0058] S24, converting the disparity value in the disparity map into a depth value in the depth map using a first formula, where the first formula is:

[0059] Among them, Z is the depth value, B is the baseline length, f is the focal length, is the parallax value.

[0060] S3, inputting the depth map into a neural network model based on deep learning for optimization to obtain an optimized depth map.

[0061] The neural network model based on deep learning includes an encoder and a decoder. The encoder includes multiple convolutional layers, each of which is connected to a ReLU activation function layer and a maximum pooling layer to gradually extract features from the input image and compress its spatial dimensions. Specifically, the encoder uses ResNet-50 as the base network and extracts multi-scale features through different layers (such as conv2_x, conv3_x, conv4_x, and conv5_x). The output of each layer of the encoder is:

[0062] in, It is The feature map of the layer, Conv represents the convolution operation, and ReLU is the activation function.

[0063] The decoder contains deconvolution layers (transposed convolution) and skip connections, which are designed to restore the original resolution while fusing multi-scale feature information. The deconvolution layer is used to gradually restore the spatial size of the feature map, while the skip connection directly connects the feature map of the corresponding level in the encoder to the corresponding layer of the decoder to help retain more local information and help the gradient flow back to the lower layer to alleviate the gradient disappearance problem. The decoder is used to restore the original resolution of the input feature map and fuse multi-scale feature information. The fused feature map is represented as:

[0064] in, is a learnable weight parameter.

[0065] Furthermore, in order to enhance the model's attention to important areas, a self-attention mechanism is introduced into the neural network model based on deep learning. The attention mechanism generates attention weights by calculating the correlation between each position and other positions, thereby enhancing the information of key areas. The attention weight is expressed as:

[0066] in, is the attention weight, Q is the query, K is the key, T is the transpose of the matrix, is the dimension of the key; Final attention output for:

[0067] Among them, V is the value matrix.

[0068] S4, input the original image into the target detection model to obtain the target category and bounding box information.

[0069] The bounding box information includes the bounding box divided by the target detection model and the horizontal and vertical coordinates of the target in the bounding box.

[0070] As an example, the target detection model can be YOLOv11. YOLOv11 integrates target detection and regression in one network, and its loss function includes classification loss , positioning loss and confidence loss :

[0071] S5, obtaining the location information of the target based on the bounding box information and the optimized depth map.

[0072] The depth information of the target is obtained through the optimized depth map, and the precise location information and the distance relative to the binocular camera can be obtained by combining the horizontal and vertical coordinates of the target with the depth information of the target. This function can accurately locate the spatial position of key facilities (such as insulators and conductors) in power operations, and is also suitable for scenarios such as autonomous driving and robot navigation.

[0073] This application uses a binocular camera to capture images from different perspectives, generates depth maps from images from different perspectives, and performs target recognition through optimized depth maps and original images. This method can not only effectively improve the speed and accuracy of distance measurement, but also provide strong support for object positioning and tracking in complex environments.

[0074] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a ranging device based on a binocular camera. The ranging device based on a binocular camera can be implemented as an independent entity or integrated in an electronic device. The electronic device can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0075] See also Figure 4 , Figure 4 The present invention specifically describes a binocular camera-based distance measuring device provided in an embodiment of the present invention, which is applied to an electronic device. The binocular camera-based distance measuring device may include: The acquisition and preprocessing module is used to acquire the original image based on the binocular camera and preprocess the original image.

[0076] The acquisition and preprocessing module includes a binocular camera group, a camera controller, a data storage unit and a preprocessor. The binocular camera group uses a pair of high-resolution cameras installed on a bracket with a fixed baseline distance, equipped with automatic white balance and exposure adjustment functions to ensure image quality; the camera controller ensures simultaneous exposure and data reading of the two cameras through an external trigger to achieve precise time synchronization; the data storage unit provides a high-speed data cache mechanism to support real-time data stream processing; the preprocessor performs image noise removal and distortion correction to improve image quality.

[0077] Figure 5 The flowchart of obtaining the original image through the binocular camera provided in the embodiment of the present application is as follows: Figure 5As shown in the figure, first, the ranging device starts and turns on the video acquisition device (binocular camera), and completes the initialization operation of the device, including the loading and calibration of camera parameters. Then, the ranging device obtains the basic information of the device and the key parameters required for image acquisition, such as camera model, resolution, exposure time, etc., to ensure that the acquisition process meets the set requirements. After the parameter configuration is completed, the ranging device starts to collect video image data. The acquisition of video images is a cyclic process. The ranging device will continuously read the input signal of the camera and record the data. Each time an acquisition is completed, the ranging device checks whether the acquisition task has reached the termination condition, such as the time length or frame number limit. If the acquisition is not completed (marked as "No"), the ranging device returns to the acquisition process to continue capturing data; if the acquisition is completed (marked as "Yes"), the acquisition process ends and the data is saved.

[0078] This process design ensures the efficiency and reliability of image data acquisition. Through the loop judgment and parameter configuration mechanism, the ranging device can adapt to different task requirements and provide high-quality raw data for subsequent image processing and depth calculation.

[0079] The depth map generation module is used to generate a disparity map based on the preprocessed original image and convert the disparity map into a depth map.

[0080] The optimization module is used to input the depth map into a neural network model based on deep learning for optimization to obtain an optimized depth map.

[0081] The optimization module includes a neural network engine and a deep data manager. The neural network engine optimizes depth estimation based on the encoder-decoder architecture and provides high-precision depth calculation. The deep data manager realizes efficient compression storage and retrieval of data.

[0082] The target detection module is used to input the original image into the target detection model to obtain the target category and bounding box information.

[0083] The positioning module is used to obtain the location information of the target based on the bounding box information and the optimized depth map.

[0084] Furthermore, the distance measurement device based on the binocular camera also includes a display module, which presents the results to the user in an intuitive manner and supports real-time viewing of the distance measurement results or exporting of measurement data.

[0085] Furthermore, the binocular camera-based distance measurement device also includes a user interaction module, which provides a user-friendly operation interface and feedback mechanism, and includes a real-time graphical display, parameter adjustment function and an audio feedback device. The user can view the depth map and the distance measurement results of the target in real time through the display screen, and adjust parameters such as resolution, illumination compensation or target selection through touch or key input. Figure 6 A user interaction flow chart provided in an embodiment of the present application, Figure 6 It describes in detail how the user operates the ranging device and the feedback mechanism of the system. The process starts with the startup of the ranging device and first enters the usability detection stage. The ranging device will check whether the required hardware equipment and software environment are running normally. When it is confirmed to be available, the user needs to decide whether to perform the calibration operation. If calibration is chosen, the ranging device will start the loop shooting mode and collect 10 sets of image data for calibration correction. This step ensures the accurate geometric calibration of the binocular camera and provides a reliable foundation for subsequent ranging.

[0086] If the user chooses to skip calibration, the shooting phase will be entered directly. The user can choose automatic mode or manual mode to shoot the target image. In automatic mode, the ranging device collects images according to preset parameters; in manual mode, the user can control the shooting process and shoot images for specific scenes or targets. After completing image acquisition, the ranging device processes the data, generates segmentation results and distance measurement results, and displays them to the user in a visual form.

[0087] Based on the segmentation and distance measurement results, users can further choose whether to view the distance information of a specific point. If the user specifies a point, the distance measurement device will extract the three-dimensional spatial position of the point, calculate its distance value, and display the result; if there is no specific point requirement, the process ends directly. The entire interactive process ensures the convenience of user operation and meets the precise distance measurement requirements in various application scenarios through clear logical control and real-time feedback.

[0088] The entire architecture realizes the complete process from data collection, deep calculation to result display through efficient collaboration between modules, meeting the requirements of high precision, real-time and flexible interaction. The device is widely used in scenarios requiring high-precision distance measurement, such as autonomous driving, robot navigation and safe distance measurement for power operations.

[0089] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0090] In addition, the embodiment of the present application further provides an electronic device, which may be a computer, a tablet computer, or other device. The electronic device may implement the steps in any embodiment of the ranging method based on a binocular camera provided in the embodiment of the present application, and therefore, may achieve the beneficial effects that can be achieved by any ranging method based on a binocular camera provided in the embodiment of the present application, as detailed in the previous embodiment, which will not be repeated here.

[0091] Figure 7 The specific structural block diagram of the electronic device provided in the embodiment of the present invention is shown, and the electronic device can be used to implement the distance measurement method based on the binocular camera provided in the above embodiment. The electronic device 500 can be a terminal, a server and other devices, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0092] The RF circuit 510 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks may use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0093] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizing the functions of taking pictures with the front camera, processing the captured images, and switching the display color of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely arranged relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0094] The input unit 530 may be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function control. The display unit 540 may be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541, which may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.

[0095] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another terminal through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earplug jack to provide communication between an external headset and the electronic device 500.

[0096] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it is not a necessary component of the electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0097] The processor 580 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580.

[0098] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 590 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0099] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors. One or more programs include instructions for performing the following operations: Acquire an original image based on a binocular camera, and preprocess the original image; Generate a disparity map based on the preprocessed original image, and convert the disparity map into a depth map; Inputting the depth map into a deep learning-based neural network model for optimization to obtain an optimized depth map; Inputting the original image into the target detection model to obtain the category and bounding box information of the target; Based on the bounding box information and the optimized depth map, the location information of the target is obtained.

[0100] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0101] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores multiple instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the ranging method based on a binocular camera provided in an embodiment of the present invention.

[0102] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0103] Since the instructions stored in the storage medium can execute the steps in any embodiment of the ranging method based on a binocular camera provided in the embodiments of the present invention, the beneficial effects that can be achieved by any ranging method based on a binocular camera provided in the embodiments of the present invention can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0104] The above is a detailed introduction to a ranging method, device, storage medium and electronic device based on a binocular camera provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A distance measurement method based on a binocular camera, characterized in that: The method comprises: Acquire an original image based on a binocular camera, and preprocess the original image; Generate a disparity map based on the preprocessed original image, and convert the disparity map into a depth map; Inputting the depth map into a deep learning-based neural network model for optimization to obtain an optimized depth map; Inputting the original image into the target detection model to obtain the category and bounding box information of the target; Based on the bounding box information and the optimized depth map, the location information of the target is obtained.

2. The distance measurement method based on a binocular camera according to claim 1, characterized in that: The original image includes a left image and a right image, and the generating of the disparity map based on the preprocessed original image includes: For each pixel point, calculating the matching cost of the pixel point in the left image and the right image; Dynamically program pixels along multiple directions to obtain aggregate costs; The disparity value of the pixel corresponding to the minimum aggregation cost is taken as the optimal disparity value, and a disparity map is constructed based on the optimal disparity value of each pixel.

3. The distance measurement method based on a binocular camera according to claim 2, characterized in that: The dynamic programming is calculated by the following formula: in, represents the current pixel position, represents parallax, represents the step length along a certain direction, represents the original cost, represents the cost after aggregation, and For penalty items.

4. The distance measurement method based on a binocular camera according to claim 1, characterized in that: Converting the disparity map into a depth map includes: The disparity value in the disparity map is converted into a depth value in the depth map by a first formula, and the first formula is: Among them, Z is the depth value, B is the baseline length, f is the focal length, is the parallax value.

5. The distance measurement method based on binocular camera according to claim 1, characterized in that: The deep learning-based neural network model includes an encoder and a decoder; The encoder includes multiple convolutional layers, each of which is connected to a ReLU activation function layer and a maximum pooling layer. The output of each layer of the encoder is: in, It is The feature map of the layer, Conv represents the convolution operation, and ReLU is the activation function; The decoder is used to restore the original resolution of the input feature map and fuse multi-scale feature information. The fused feature map is expressed as: in, is a learnable weight parameter.

6. The distance measurement method based on binocular camera according to claim 5, characterized in that: The method further comprises: A self-attention mechanism is introduced into the neural network model based on deep learning, and the attention weight is expressed as: in, is the attention weight, Q is the query, K is the key, T is the transpose of the matrix, is the dimension of the key; Final attention output for: Among them, V is the value matrix.

7. The distance measurement method based on binocular camera according to claim 1, characterized in that: Preprocessing the original image includes: The original image is subjected to denoising and edge preservation processing by bilateral filtering, and the formula corresponding to the bilateral filtering is: in, Pixel The strength value of is the spatial Gaussian weight, based on the spatial distance of pixels, is the pixel intensity difference weight; is a normalization factor used to ensure that the filtered value is within a reasonable range.

8. A distance measuring device based on a binocular camera, characterized in that: include: An acquisition and preprocessing module, used to acquire original images based on a binocular camera and preprocess the original images; A depth map generation module, used to generate a disparity map based on the preprocessed original image, and convert the disparity map into a depth map; An optimization module, used for inputting the depth map into a neural network model based on deep learning for optimization, to obtain an optimized depth map; An object detection module is used to input the original image into an object detection model to obtain the category and bounding box information of the object; A positioning module is used to obtain the position information of the target based on the bounding box information and the optimized depth map.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the ranging method based on a binocular camera as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the ranging method based on a binocular camera as described in any one of claims 1 to 7.

Images