Panoramic-dashcam-based method and system for improving resolution of recorded pictures, and electronic device and medium

Abstract

A panoramic-dashcam-based method and system for improving the resolution of recorded pictures, and an electronic device and a medium. The method comprises: identifying illumination features of a driving recording scenario, and analyzing an illumination state of the driving recording scenario (S1); identifying an illumination influence coefficient of the illumination state for a panoramic dashcam, and constructing an illumination equalization network of the panoramic dashcam (S2); using the panoramic dashcam to perform driving recording, so as to obtain a driving recording video, and performing frame extraction on the driving recording video, so as to obtain driving recording frame images (S3); identifying a low-resolution image from the driving recording frame images, extracting image features of the low-resolution image, performing mapping on the image features so as to obtain a mapping relationship, and performing high-resolution reconstruction on the low-resolution image so as to obtain a high-resolution image (S4); and constructing a high-resolution driving recording video of the driving recording video (S5).

Description

Method, system, electronic device and medium for improving the resolution of recorded images based on panoramic driving recorder

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 25, 2023, with application number 202311788192.2, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of video processing, and for example, to a method, system, electronic device, and medium for improving the resolution of recorded images based on a panoramic driving recorder. Background Art

[0003] Recording resolution generally refers to the clarity and level of detail of a scene captured by a video or image sensor. This resolution, defined as the number of pixels per unit length (usually an inch), determines image quality. Higher resolution means more pixels per unit length, resulting in finer image quality.

[0004] The improvement of recorded image resolution is mainly achieved by selecting high-resolution sensors and replacing large-aperture lenses. This method mainly optimizes the hardware equipment. When encountering overexposed or dark images, details will be lost, resulting in low-resolution images in the video recorded by the dash cam, resulting in poor improvement in recorded image resolution.

[0005] Summary of the Invention

[0006] The present application provides a method, system, electronic device and medium for improving the resolution of recorded images based on a panoramic driving recorder, which can improve the optimization effect of the resolution of recorded images of the panoramic driving recorder.

[0007] This application provides a method for improving the resolution of recorded images based on a panoramic driving recorder, comprising:

[0008] Acquiring a driving recording scene of a panoramic driving recorder, identifying lighting characteristics of the driving recording scene, and analyzing the lighting state of the driving recording scene based on the lighting characteristics;

[0009] Identifying an illumination influence coefficient of the illumination state on the panoramic driving recorder, and constructing a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient;

[0010] Based on the balanced illumination network, using the panoramic driving recorder to record driving to obtain a driving record video, and framing the driving record video to obtain a driving record frame image;

[0011] Identifying a low-resolution image of the driving record frame image, extracting image features of the low-resolution image, mapping the image features to obtain a mapping relationship, and reconstructing the low-resolution image at a high resolution based on the mapping relationship to obtain a high-resolution image;

[0012] A high-resolution driving record video of the driving record video is constructed based on the high-resolution image.

[0013] Optionally, identifying the lighting features of the driving recording scene includes:

[0014] Acquiring a scene image of the driving record scene;

[0015] Calculating the third-order moment of image brightness of the scene image;

[0016] Analyzing the illumination distribution of the driving recording scene based on the third-order moment of the image brightness;

[0017] Based on the illumination distribution, illumination features of the driving recording scene are extracted.

[0018] Optionally, calculating the third-order moment of image brightness of the scene image includes:

[0019] Marking image pixels of the scene image;

[0020] Based on the image pixels, the third-order moment of the image brightness of the scene image is calculated using the following formula: m30 = ∑∑a^3*f(a,c)m03 = ∑∑c^3*f(a,c)m12 = ∑∑a^2*c*f(a,c)

[0021] Among them, m30 represents the third-order moment of the image brightness of the scene image in the x-axis direction, a represents the horizontal coordinate of the image pixel point, c represents the vertical coordinate of the image pixel point, f(a,c) represents the pixel value of the image pixel point, m03 represents the third-order moment of the image brightness of the scene image in the y-axis direction, and m12 represents the second-order mixed moment of the scene image in the x-axis and y-axis directions.

[0022] Optionally, analyzing the illumination state of the driving recording scene based on the illumination feature includes:

[0023] Training a lighting recognition model for the driving recording scene based on the lighting features and the scene image corresponding to the driving recording scene;

[0024] Calculating a model recall rate of the illumination recognition model;

[0025] When the model recall rate meets the requirement, the illumination state of the driving record scene is identified using the illumination recognition model.

[0026] Optionally, the identifying the illumination influence coefficient of the illumination state on the panoramic driving recorder includes:

[0027] Identifying the lighting influencing factors of the panoramic driving recorder;

[0028] Analyzing the relationship between the illumination influencing factors and the illumination influence of the panoramic driving recorder;

[0029] quantifying the illumination impact factor based on the illumination state to obtain a quantized illumination impact factor;

[0030] Based on the illumination influence relationship and the quantified illumination influence factor, an illumination influence coefficient of the panoramic driving recorder is calculated.

[0031] Optionally, the calculating the illumination impact coefficient of the panoramic driving recorder based on the illumination impact relationship and the quantified illumination impact factor includes:

[0032] Based on the illumination influence relationship, determining an influence factor weight of the panoramic driving recorder corresponding to the illumination influence factor;

[0033] Based on the impact factor weight and the quantified illumination impact factor, the illumination impact coefficient of the panoramic driving recorder is calculated using the following formula:

[0034] Among them, τ represents the illumination influence coefficient of the panoramic driving recorder, m represents the number of illumination influence factors corresponding to the panoramic driving recorder, and E i It represents the quantized illumination impact factor of the panoramic driving recorder corresponding to the i-th illumination impact factor, Q i represents the weight of the influence factor of the panoramic driving recorder corresponding to the i-th illumination influence factor, and σ represents the mutual influence coefficient of the panoramic driving recorder corresponding to the i-th illumination influence factor on other illumination influence factors.

[0035] Optionally, extracting image features of the low-resolution image includes:

[0036] Converting the low-resolution image into a grayscale image;

[0037] Dividing the grayscale image into image blocks;

[0038] Converting the image block into a binary image;

[0039] extracting binary image features of the binary image;

[0040] Performing dimensionality reduction processing on the binary image features to obtain image features of the low-resolution image.

[0041] Optionally, converting the image block into a binary image includes:

[0042] Mark the center pixel of the image block; calculate the local binary pattern value of the center pixel using the following formula: LBP(r,v)=∑(b=0to7)[U(gradient(r,v,b))]

[0043] Where LBP(r,v) represents the local binary pattern value of the center pixel, U represents the sign function, gradient(r,v,b) represents the gradient value between the center pixel (r,v) and the neighboring pixel (b), r represents the horizontal coordinate of the center pixel, and v represents the vertical coordinate of the center pixel;

[0044] A binary image of the image block is constructed based on the local binary pattern value.

[0045] Optionally, reconstructing the low-resolution image with high resolution based on the mapping relationship to obtain a high-resolution image includes:

[0046] Based on the mapping relationship, reconstructing the image block corresponding to the low-resolution image at high resolution to obtain a high-resolution image block;

[0047] Fusing the high-resolution image blocks to obtain the fused high-resolution image blocks;

[0048] The fused high-resolution image block is smoothed to obtain the high-resolution image.

[0049] The present application also provides a system for improving the resolution of recorded images based on a panoramic driving recorder, the system comprising:

[0050] a lighting state analysis module configured to obtain a driving recording scene of the panoramic driving recorder, identify lighting characteristics of the driving recording scene, and analyze the lighting state of the driving recording scene based on the lighting characteristics;

[0051] an illumination balancing module configured to identify an illumination influence coefficient of the illumination state on the panoramic driving recorder, and construct a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient;

[0052] a video imaging module configured to record driving using the panoramic driving recorder based on the balanced illumination network to obtain a driving record video, and to frame the driving record video to obtain a driving record frame image;

[0053] a high-resolution reconstruction module configured to identify a low-resolution image of the driving record frame image, extract image features of the low-resolution image, map the image features to obtain a mapping relationship, and perform high-resolution reconstruction on the low-resolution image based on the mapping relationship to obtain a high-resolution image;

[0054] The image video module is configured to construct a high-resolution driving record video of the driving record video based on the high-resolution image.

[0055] The present application provides an electronic device, including a memory, a processor, a communication bus, a communication interface, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for improving the resolution of recorded images based on a panoramic driving recorder is implemented.

[0056] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for improving the resolution of recorded images based on a panoramic driving recorder is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] FIG1 is a flow chart of a method for improving the resolution of recorded images based on a panoramic driving recorder according to an embodiment of the present application;

[0058] FIG2 is a functional module diagram of a system for improving the resolution of recorded images based on a panoramic driving recorder according to an embodiment of the present application;

[0059] FIG3 is a schematic structural diagram of an electronic device of a system for improving the resolution of recorded images based on a panoramic driving recorder provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] The embodiment of the present application provides a method for improving the resolution of recorded images based on a panoramic driving recorder. The execution subject of the method for improving the resolution of recorded images based on a panoramic driving recorder includes at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for improving the resolution of recorded images based on a panoramic driving recorder can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0061] 1 is a flow chart of a method for improving the resolution of recorded images based on a panoramic driving recorder according to an embodiment of the present application. In this embodiment, the method for improving the resolution of recorded images based on a panoramic driving recorder includes the following steps.

[0062] S1. Acquire a driving recording scene of a panoramic driving recorder, identify lighting features of the driving recording scene, and analyze the lighting state of the driving recording scene based on the lighting features.

[0063] In the embodiment of the present application, the driving recording scene refers to the scene recorded by the panoramic driving recorder corresponding to the moving vehicle, such as driving on urban roads, driving on highways, driving at night, driving under unconventional driving conditions, etc.

[0064] By identifying the lighting characteristics of the recorded driving scene, embodiments of the present application can reflect the lighting conditions and scene attributes of the image, providing a data foundation for subsequent image recognition, analysis, and understanding. The lighting characteristics refer to characteristic attributes of the lighting in the recorded driving scene, such as persistent darkness, partial exposure, and other characteristic attributes.

[0065] As an embodiment of the present application, the identifying of the lighting characteristics of the driving recording scene includes: obtaining a scene image of the driving recording scene; calculating the third-order moment of image brightness of the scene image; analyzing the lighting distribution of the driving recording scene based on the third-order moment of image brightness; and extracting the lighting characteristics of the driving recording scene based on the lighting distribution.

[0066] The scene image refers to the image of the current scene in the driving record, the third-order moment of image brightness refers to a moment describing the image brightness distribution, and the illumination distribution refers to analyzing the brightness distribution of the image, for example, the overall brightness and darkness, brightness dispersion, brightness symmetry, and other distribution conditions.

[0067] Optionally, as an optional embodiment of the present application, calculating the third-order moment of the image brightness of the scene image includes: marking image pixels of the scene image; and calculating the third-order moment of the image brightness of the scene image based on the image pixels using the following formula: m30 = ∑∑a^3*f(a,c)m03 = ∑∑c^3*f(a,c)m12 = ∑∑a^2*c*f(a,c)

[0068] Among them, m30 represents the third-order moment of the image brightness of the scene image in the x-axis direction, a represents the horizontal coordinate of the image pixel point, c represents the vertical coordinate of the image pixel point, f(a,c) represents the pixel value of the image pixel point, m03 represents the third-order moment of the image brightness of the scene image in the y-axis direction, and m12 represents the second-order mixed moment of the scene image in the x-axis and y-axis directions.

[0069] The second-order mixed moment refers to a moment that describes the degree of correlation between the third-order moment of the image brightness of the scene image in the x-axis direction and the third-order moment of the image brightness of the scene image in the y-axis direction.

[0070] In the embodiment of the present application, based on the illumination characteristics, the illumination state of the driving recording scene can be analyzed to later analyze whether the current illumination can achieve recording by the driving recorder. The illumination state refers to the current illumination condition of the driving recording scene, such as whether the recording environment is too dark or the recording environment is overexposed.

[0071] As an embodiment of the present application, the analysis of the lighting state of the driving recording scene based on the lighting features includes: training a lighting recognition model of the driving recording scene based on the lighting features and the scene image corresponding to the driving recording scene; calculating the model recall rate of the lighting recognition model; when the model recall rate meets the requirements, using the lighting recognition model to identify the lighting state of the driving recording scene.

[0072] The illumination recognition model refers to a model that can identify the illumination state within an image. The illumination recognition model can be trained using a deep learning model after obtaining the extracted illumination features. Commonly used models include linear classifiers, support vector machines (SVMs), decision trees, and the like. During training, the model is optimized for accurate recognition of target objects by optimizing the loss function. The model recall rate refers to the accuracy of the illumination recognition model's recognition results.

[0073] S2. Identify the illumination influence coefficient of the illumination state on the panoramic driving recorder, and construct a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient.

[0074] In the embodiment of the present application, the illumination influence coefficient of the illumination state on the panoramic driving recorder can be identified to analyze the degree of influence of the current illumination on the driving recorder, thereby providing a basis for reducing the illumination influence in the future. The illumination influence coefficient refers to the degree of influence of the current illumination condition on the driving record of the panoramic driving recorder.

[0075] Optionally, as an embodiment of the present application, the identifying the lighting impact coefficient of the lighting state on the panoramic driving recorder includes: identifying the lighting impact factor of the panoramic driving recorder; analyzing the lighting impact relationship of the lighting impact factor on the panoramic driving recorder; quantifying the lighting impact factor based on the lighting state to obtain a quantized lighting impact factor; and calculating the lighting impact coefficient of the panoramic driving recorder based on the lighting impact relationship and the quantified lighting impact factor.

[0076] Among them, the lighting impact factor refers to the lighting factors that affect the recording of the panoramic driving recorder, such as environmental exposure, too dark environment, etc. The lighting impact relationship refers to the impact of the lighting impact factor on the picture recorded by the panoramic driving recorder. For example, the more severe the exposure, the unclear the picture recorded by the panoramic driving recorder. The quantified lighting impact factor refers to the quantified value of the lighting impact factor, such as exposure, brightness, etc.

[0077] Optionally, as an optional embodiment of the present application, calculating the illumination impact coefficient of the panoramic driving recorder based on the illumination impact relationship and the quantified illumination impact factor includes: determining an influence factor weight of the panoramic driving recorder corresponding to the illumination impact factor based on the illumination impact relationship; and calculating the illumination impact coefficient of the panoramic driving recorder based on the influence factor weight and the quantified illumination impact factor using the following formula:

[0078] Among them, τ represents the illumination influence coefficient of the panoramic driving recorder, m represents the number of illumination influence factors corresponding to the panoramic driving recorder, and E i It represents the quantized illumination impact factor of the panoramic driving recorder corresponding to the i-th illumination impact factor, Q i represents the weight of the influencing factor of the panoramic driving recorder corresponding to the i-th illumination influencing factor, and σ represents the mutual influence coefficient of the panoramic driving recorder corresponding to the i-th illumination influencing factor on other illumination influencing factors.

[0079] Among them, the influencing factor weight refers to the degree of influence of the lighting influencing factor on the panoramic driving recorder image recording, and the factor mutual influence coefficient refers to the degree of influence of one of the lighting influencing factors on the other lighting influencing factors. The quantified lighting influencing factors can be tabulated to extract the numerical relationship between one of the lighting influencing factors and the other lighting influencing factors, thereby analyzing the factor mutual influence coefficient of the lighting influencing factor.

[0080] In the embodiment of the present application, based on the illumination influence coefficient, a balanced illumination network is constructed for the panoramic dash cam, which enables the panoramic dash cam to record driving in special environments and improves the image resolution. The balanced illumination network refers to a network that adjusts the illumination of the panoramic dash cam in special lighting environments, and the balanced illumination network can be implemented using infrared technology.

[0081] S3. Based on the balanced illumination network, use the panoramic driving recorder to record driving to obtain a driving record video, and frame the driving record video to obtain a driving record frame image.

[0082] Optionally, in the embodiment of the present application, based on the balanced illumination network, the panoramic driving recorder is used to record driving, and the driving record video obtained can realize the driving record of the vehicle. Wherein, the driving record video refers to the video of the vehicle driving process recorded by the panoramic driving recorder.

[0083] In the embodiment of the present application, the driving record video is framed to obtain driving record frame images, which can be used to visualize the video, more accurately identify and repair low-resolution images, thereby improving the resolution of the recorded image. The driving record frame images refer to a collection of images obtained by decomposing the driving record video into frames. The framing of the driving record video can be processed using the OpenCV library in Python.

[0084] S4. Identify the low-resolution image of the driving record frame image, extract image features of the low-resolution image, map the image features to obtain a mapping relationship, and reconstruct the low-resolution image at a high resolution based on the mapping relationship to obtain a high-resolution image.

[0085] In the embodiment of the present application, by identifying low-resolution images in the dash cam frame images, images that do not meet the resolution requirements can be screened out and resolution improved. The low-resolution images are images in the dash cam frame images that do not meet the image quality requirements. The low-resolution images identified in the dash cam frame images can be analyzed for image quality using tools such as ImageJ and FFmpeg.

[0086] The embodiment of the present application can provide data basis for subsequent image reconstruction by extracting image features of the low-resolution image, wherein the image features refer to characteristic attributes of the low-resolution image, such as texture features, global features, and other characteristic attributes.

[0087] As an embodiment of the present application, the extracting of image features of the low-resolution image includes: converting the low-resolution image into a grayscale image; dividing the grayscale image into image blocks; converting the image blocks into binary images; extracting binary image features of the binary image; and performing dimensionality reduction processing on the binary image features to obtain image features of the low-resolution image.

[0088] Among them, the grayscale image refers to the image after the color of the low-resolution image is converted into gray, the image block refers to the local image block into which the grayscale image is divided, the binary image refers to a digital image in which each pixel has only two possible values ​​or grayscale levels, and the binary image feature refers to a characteristic attribute that describes the image information of the binary image. For example, the binary image feature is a feature operator of a local binary pattern that describes the local texture of the image.

[0089] Optionally, as an optional embodiment of the present application, converting the image block into a binary image includes: marking a central pixel of the image block; and calculating a local binary pattern value of the central pixel using the following formula: LBP(r,v)=∑(b=0 to 7)[U(gradient(r,v,b))]

[0090] Where LBP(r,v) represents the local binary pattern value of the center pixel, U represents the sign function, gradient(r,v,b) represents the gradient value between the center pixel (r,v) and the neighboring pixel (b), r represents the horizontal coordinate of the center pixel, and v represents the vertical coordinate of the center pixel;

[0091] A binary image of the image block is constructed based on the local binary pattern value.

[0092] The local binary pattern value refers to the texture information within the neighborhood of the pixel. For example, a neighborhood (e.g., an 8-neighborhood) is taken with the central pixel as the center in the image block, and the brightness value of the pixel is compared with that of other pixels in the neighborhood. If the brightness value of the central pixel is greater than that of a pixel in the neighborhood, the pixel is marked as 1, otherwise it is marked as 0. In this way, for an 8-neighborhood, an 8-bit binary number is obtained, such as 11001111, which is the local binary pattern value of the pixel. The sign function returns 1 when the gradient value is greater than 0, otherwise it returns 0.

[0093] The embodiment of the present application maps the image features to obtain a mapping relationship, which provides a basis for high-resolution reconstruction of the image in the later stage. The mapping relationship refers to the relationship between the low-resolution image and the high-resolution image obtained after mapping the image features to a nonlinear space. The mapping of the image features can be achieved by a convolutional neural network (CNN) model, wherein the CNN model consists of a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer is used to extract local features of the image, the pooling layer is used to reduce the dimension of the feature map, and the fully connected layer is used to map the feature map to the final classification or regression result.

[0094] Optionally, in the embodiment of the present application, the low-resolution image is reconstructed at a high resolution based on the mapping relationship to obtain a high-resolution image, thereby improving the resolution of the image and thus improving the clarity of the driving recording. The high-resolution image refers to an image obtained by improving the quality of the low-resolution image.

[0095] Optionally, as an embodiment of the present application, the high-resolution reconstruction of the low-resolution image based on the mapping relationship to obtain a high-resolution image includes: based on the mapping relationship, high-resolution reconstruction of the image blocks corresponding to the low-resolution image to obtain high-resolution image blocks; fusing the high-resolution image blocks to obtain fused high-resolution image blocks; and smoothing the fused high-resolution image blocks to obtain the high-resolution image.

[0096] The high-resolution image block refers to the image block obtained by performing high-resolution reconstruction on the image block, and the fused high-resolution image block refers to a set of image blocks obtained by fusion of the high-resolution image blocks.

[0097] S5. Construct a high-resolution driving record video of the driving record video based on the high-resolution image.

[0098] In the embodiment of the present application, by constructing a high-resolution dash cam video based on the high-resolution image, the resolution of the dash cam video can be increased. The high-resolution dash cam video refers to a dash cam video with its image resolution increased. The construction of the high-resolution dash cam video can be achieved using software such as Adobe Premiere, Final Cut Pro, or iMovie.

[0099] The embodiment of the present application can reflect the lighting conditions and scene attributes of the image by identifying the lighting characteristics of the driving record scene, which provides a data basis for the recognition, analysis and understanding of the subsequent image; the embodiment of the present application analyzes the lighting state of the driving record scene based on the lighting characteristics, which can be used for the subsequent analysis of whether the current lighting can achieve the recording of the driving recorder; the embodiment of the present application identifies the lighting influence coefficient of the lighting state on the panoramic driving recorder, which can analyze the degree of influence of the current lighting on the driving recorder, thereby providing a basis for reducing the lighting influence in the subsequent period; the embodiment of the present application can filter out images that do not meet the resolution requirements by identifying the low-resolution images of the driving record frame images, and improve the resolution. The embodiment of the present application maps the image features to obtain a mapping relationship, which provides a basis for the subsequent high-resolution reconstruction of the image. Wherein, the mapping relationship refers to the relationship between the low-resolution image and the high-resolution image obtained after mapping the image features to a nonlinear space. The embodiment of the present application can improve the resolution of the driving record video by constructing a high-resolution driving record video of the driving record video based on the high-resolution image. Therefore, the method for improving the resolution of recorded images based on a panoramic driving recorder provided in this application can improve the optimization effect of the resolution of recorded images of the panoramic driving recorder.

[0100] As shown in FIG2 , it is a functional module diagram of a system for improving the resolution of recorded images based on a panoramic driving recorder provided in one embodiment of the present application.

[0101] The panoramic dashcam-based recorded image resolution enhancement system 200 described in this application can be installed in an electronic device. Depending on the functions implemented, the panoramic dashcam-based recorded image resolution enhancement system 200 may include an illumination state analysis module 201, an illumination balancing module 202, a video visualization module 203, a high-resolution reconstruction module 204, and an image visualization module 205. The modules described in this application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, and are stored in the memory of the electronic device.

[0102] In this embodiment, the functions of the multiple modules / units are as follows:

[0103] The illumination state analysis module 201 is configured to obtain a driving recording scene of the panoramic driving recorder, identify illumination features of the driving recording scene, and analyze the illumination state of the driving recording scene based on the illumination features;

[0104] The illumination balancing module 202 is configured to identify an illumination influence coefficient of the illumination state on the panoramic driving recorder, and construct a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient;

[0105] The video imaging module 203 is configured to use the panoramic driving recorder to record driving based on the balanced illumination network to obtain a driving record video, and to frame the driving record video to obtain a driving record frame image;

[0106] The high-resolution reconstruction module 204 is configured to identify a low-resolution image of the driving record frame image, extract image features of the low-resolution image, map the image features to obtain a mapping relationship, and perform high-resolution reconstruction on the low-resolution image based on the mapping relationship to obtain a high-resolution image;

[0107] The image video module 205 is configured to construct a high-resolution driving record video of the driving record video based on the high-resolution image.

[0108] The multiple modules described in the system 200 for improving the recording image resolution based on a panoramic driving recorder described in the embodiment of the present application adopt the same technical means as the method for improving the recording image resolution based on a panoramic driving recorder described in the accompanying drawings when used, and can produce the same technical effects, which will not be repeated here.

[0109] An embodiment of the present application provides an electronic device for implementing a method for improving the resolution of recorded images based on a panoramic driving recorder.

[0110] As shown in Figure 3, the electronic device may include a processor 30, a memory 31, a communication bus 32 and a communication interface 33, and may also include a computer program stored in the memory 31 and run on the processor 30, such as a method program for improving the resolution of recorded images based on a panoramic driving recorder.

[0111] In some embodiments, the processor 30 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 30 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing programs or modules stored in the memory 31 (such as executing a program for enhancing the resolution of recorded images based on a panoramic driving recorder) and calling data stored in the memory 31 to perform various functions of the electronic device and process data.

[0112] The memory 31 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, a secure digital (SD) or data register (DX) memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 31 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 31 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (SMC), an SD card, a flash card, etc. equipped on the electronic device. The memory 31 can also include both an internal storage unit and an external storage device of the electronic device. The memory 31 can be configured not only to store application software installed in the electronic device and various types of data, such as the code of a program for enhancing the resolution of recorded images based on a panoramic driving recorder, but can also be configured to temporarily store data that has been output or is to be output.

[0113] The communication bus 32 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 31 and at least one processor 30, etc.

[0114] The communication interface 33 is configured for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a wireless fidelity (WI-FI) interface, a Bluetooth interface, etc.), which is usually configured to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be a light-emitting diode (LED) display, a liquid crystal display, a touch-sensitive liquid crystal display, and an organic light-emitting diode (OLED) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is configured to display information processed in the electronic device and to display a visual user interface.

[0115] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering multiple components. The power source may be logically connected to the at least one processor 30 via a power management system, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management system. The power source may further include any components such as one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0116] The embodiment is for illustration purposes only and the scope of the patent application is not limited to this structure.

[0117] The program for improving the resolution of the recorded images based on the panoramic driving recorder stored in the memory 31 of the electronic device is a combination of multiple instructions. When running in the processor 30, it can achieve the following:

[0118] Acquiring a driving recording scene of a panoramic driving recorder, identifying lighting characteristics of the driving recording scene, and analyzing the lighting state of the driving recording scene based on the lighting characteristics;

[0119] Identifying an illumination influence coefficient of the illumination state on the panoramic driving recorder, and constructing a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient;

[0120] Based on the balanced illumination network, using the panoramic driving recorder to record driving to obtain a driving record video, and framing the driving record video to obtain a driving record frame image;

[0121] Identifying a low-resolution image of the driving record frame image, extracting image features of the low-resolution image, mapping the image features to obtain a mapping relationship, and reconstructing the low-resolution image at a high resolution based on the mapping relationship to obtain a high-resolution image;

[0122] A high-resolution driving record video of the driving record video is constructed based on the high-resolution image.

[0123] The method for the processor 30 to implement the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the drawings, which will not be repeated here.

[0124] If the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system that can carry the computer program code, a recording medium, a Universal Serial Bus flash disk (U disk), a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM). The storage medium may be a non-transitory storage medium.

[0125] The present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0126] Acquiring a driving recording scene of a panoramic driving recorder, identifying lighting characteristics of the driving recording scene, and analyzing the lighting state of the driving recording scene based on the lighting characteristics;

[0127] Identifying an illumination influence coefficient of the illumination state on the panoramic driving recorder, and constructing a balanced illumination network for the panoramic driving recorder based on the illumination influence coefficient;

[0128] Based on the balanced illumination network, using the panoramic driving recorder to record driving to obtain a driving record video, and framing the driving record video to obtain a driving record frame image;

[0129] Identifying a low-resolution image of the driving record frame image, extracting image features of the low-resolution image, mapping the image features to obtain a mapping relationship, and reconstructing the low-resolution image at a high resolution based on the mapping relationship to obtain a high-resolution image;

[0130] A high-resolution driving record video of the driving record video is constructed based on the high-resolution image.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0132] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0133] In addition, multiple functional modules in various embodiments of the present application may be integrated into a single processing unit, or multiple units may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0134] The present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other forms.

[0135] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0136] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0137] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems recited in a system claim may also be implemented by a single unit or system through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

Claims

1. A method for improving the resolution of recorded images based on a panoramic driving recorder, comprising: Obtaining the driving record scenario of the panoramic driving recorder, identifying the lighting characteristics of the driving record scenario, and analyzing the lighting state of the driving record scenario based on the lighting characteristics; Identifying the lighting influence coefficient of the lighting state on the panoramic driving recorder, and constructing an equalized lighting network of the panoramic driving recorder based on the lighting influence coefficient; Based on the equalized lighting network, using the panoramic driving recorder to perform driving record to obtain a driving record video, and frame the driving record video to obtain driving record frame images; Identifying the low-resolution images in the driving record frame images, extracting the image features of the low-resolution images, mapping the image features to obtain a mapping relationship, and performing high-resolution reconstruction on the low-resolution images based on the mapping relationship to obtain high-resolution images; Based on the high-resolution images, constructing a high-resolution driving record video of the driving record video.

2. The method according to claim 1, wherein The identifying the lighting characteristics of the driving record scenario includes: Obtaining the scenario image of the driving record scenario; Calculating the third-order moment of the image brightness of the scenario image; Analyzing the lighting distribution of the driving record scenario based on the third-order moment of the image brightness; Extracting the lighting characteristics of the driving record scenario based on the lighting distribution.

3. The method according to claim 2, wherein, The calculating the third-order moment of the image brightness of the scenario image includes: Marking the image pixel points of the scenario image; Based on the image pixel points, using the following formula to calculate the third-order moment of the image brightness of the scenario image: m30 = ∑∑a^3 * f(a, c) m03 = ∑∑c^3 * f(a, c) m12 = ∑∑a^2 * c * f(a, c) Where, m30 represents the third-order moment of the image brightness of the scenario image in the x-axis direction, a represents the abscissa of the image pixel point, c represents the ordinate of the image pixel point, f(a, c) represents the pixel value of the image pixel point, m03 represents the third-order moment of the image brightness of the scenario image in the y-axis direction, and m12 represents the second-order mixed moment of the scenario image in the x-axis and y-axis directions.

4. The method according to claim 1, wherein, The analyzing the lighting state of the driving record scenario based on the lighting characteristics includes: Training a lighting recognition model of the driving record scenario based on the lighting characteristics and the scenario image corresponding to the driving record scenario; Calculating the model recall rate of the lighting recognition model; In response to the model recall rate meeting the requirements, using the lighting recognition model to identify the lighting state of the driving record scenario.

5. The method according to claim 1, wherein, The identifying the lighting influence coefficient of the lighting state on the panoramic driving recorder includes: Identifying the lighting influence factors of the panoramic driving recorder; Analyzing the lighting influence relationship of the lighting influence factors on the panoramic driving recorder; Quantifying the lighting influence factors based on the lighting state to obtain quantified lighting influence factors; Calculating the lighting influence coefficient of the panoramic driving recorder based on the lighting influence relationship and the quantified lighting influence factors.

6. The method according to claim 5, wherein, Calculating the light influence coefficient of the panoramic driving recorder based on the light influence relationship and the quantified light influence factor includes: Based on the light influence relationship, determining the influence factor weight of the light influence factor corresponding to the panoramic driving recorder; Based on the influence factor weight and the quantified light influence factor, the light influence coefficient of the panoramic driving recorder is calculated using the following formula: Among them, τ represents the light influence coefficient of the panoramic driving recorder, m represents the number of light influence factors corresponding to the panoramic driving recorder, E i represents the quantified light influence factor of the i-th light influence factor corresponding to the panoramic driving recorder, Q i represents the influence factor weight of the i-th light influence factor corresponding to the panoramic driving recorder, and σ represents the factor interaction coefficient of the i-th light influence factor corresponding to the panoramic driving recorder on other light influence factors.

7. The method according to claim 1, wherein The extracting the image features of the low-resolution image includes: Converting the low-resolution image into a grayscale image; Dividing the grayscale image into image blocks; Converting the image block into a binary image; Extracting the binary image features of the binary image; Performing dimensionality reduction processing on the binary image features to obtain the image features of the low-resolution image.

8. The method according to claim 7, wherein, The converting the image block into a binary image includes: Marking the central pixel of the image block; calculating the local binary pattern value of the central pixel using the following formula: LBP(r, v) = ∑(b = 0 to 7)[U(gradient(r, v, b))] where LBP(r, v) represents the local binary pattern value of the central pixel, U represents the sign function, gradient(r, v, b) represents the gradient value between the central pixel (r, v) and the neighborhood pixel (b), r represents the abscissa of the central pixel, and v represents the ordinate of the central pixel; Based on the local binary pattern value, constructing the binary image of the image block.

9. The method according to claim 1, wherein The performing high-resolution reconstruction on the low-resolution image based on the mapping relationship to obtain a high-resolution image includes: Based on the mapping relationship, performing high-resolution reconstruction on the image block corresponding to the low-resolution image to obtain a high-resolution image block; Fusing the high-resolution image blocks to obtain a fused high-resolution image block; Performing smoothing processing on the fused high-resolution image block to obtain the high-resolution image.

10. A recording picture resolution improvement system based on a panoramic driving recorder, wherein, A system for executing the method for improving the resolution of the recording screen based on a panoramic driving recorder according to any one of claims 1 to 9, the system includes: A light state analysis module, configured to obtain the driving record scene of the panoramic driving recorder, identify the light characteristics of the driving record scene, and analyze the light state of the driving record scene based on the light characteristics; A light balance module, configured to identify the light influence coefficient of the panoramic driving recorder on the light state, and construct an equilibrium light network of the panoramic driving recorder based on the light influence coefficient; A video imaging module, configured to perform driving recording using the panoramic driving recorder based on the equilibrium light network to obtain a driving recording video, and frame the driving recording video to obtain a driving recording frame image; A high-resolution reconstruction module, configured to identify the low-resolution image of the driving recording frame image, extract the image features of the low-resolution image, map the image features to obtain a mapping relationship, and perform high-resolution reconstruction on the low-resolution image based on the mapping relationship to obtain a high-resolution image; An image video module, configured to construct a high-resolution driving recording video of the driving recording video based on the high-resolution image.

11. An electronic device, comprising: A memory, a processor, a communication bus, a communication interface, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the computer program, the method for improving the resolution of the recording screen based on a panoramic driving recorder according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, the method for improving the resolution of the recording screen based on a panoramic driving recorder according to any one of claims 1 to 9 is implemented.

Images