A portrait processing method, device, electronic equipment and medium

Abstract

The embodiments of the present application provide a portrait processing method, device, electronic device and medium, which belong to the field of computer technology. The method is applied to new energy vehicles, including: detecting the current state information of the new energy vehicle; controlling the on-board camera to obtain the driver's image information according to the state information; and denoising the image information, and judging the driver's driving state based on the denoised image information. Applying the technical solution of the present application can more effectively avoid the problem of image blur caused by vehicle shaking, and improve the imaging quality.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method, device, electronic device and medium for processing a human portrait. Background Art

[0002] With the development of automobile intelligence, the application of vehicle-mounted cameras is becoming more and more widespread, and the application of portrait processing technology in automobiles is becoming more and more important. Portrait processing technology mainly analyzes and processes the images obtained by the camera to achieve human recognition and tracking. In automobiles, portrait processing technology can be used for driver monitoring, passenger monitoring, driver behavior analysis, etc. to improve driving safety and comfort. However, since the car will shake during driving and the driver is limited by the driving state and cannot focus on shooting, the acquired images are often blurred and jittery, affecting the imaging quality.

[0003] To solve this problem, existing technologies mainly use hardware devices (such as stabilizers) to reduce the shaking of the car, or use software algorithms to denoise and de-jitter the acquired images. For example, some high-end cars are equipped with optical stabilizers, which can effectively reduce camera shaking and improve image quality. At the same time, there are also some software algorithms that can denoise and de-jitter the image by analyzing the motion information of the image to improve the image quality.

[0004] However, there are still some problems with existing technologies in practical applications. First, the price of hardware devices (such as stabilizers) is high, which increases the cost of the car. Second, the processing effect of the software algorithm is affected by many factors, such as the complexity of the image, the efficiency of the algorithm, etc., and the processing effect may not be satisfactory. In addition, existing technologies often cannot adjust the processing strategy according to the state of the car (such as speed, direction, etc.), resulting in unstable quality of the processed image. Summary of the invention

[0005] The present application provides a portrait processing method, device, electronic device and medium, aiming to improve the above-mentioned problems.

[0006] In a first aspect, the present application provides a portrait processing method, which is applied to new energy vehicles, and the method comprises:

[0007] Detecting the current status information of the new energy vehicle;

[0008] Controlling the vehicle-mounted camera to obtain image information of the driver according to the status information; and

[0009] The image information is subjected to denoising processing, and the driving state of the driver is determined based on the denoised image information.

[0010] In one embodiment, the state information includes at least one of speed, direction, and displacement travel of a shock absorber in a vehicle suspension.

[0011] In one embodiment, before the step of performing denoising on the image information, the method further comprises:

[0012] De-jittering is performed on the image information.

[0013] In one embodiment, it also includes:

[0014] The strength of the denoising process and the de-jitter process is dynamically adjusted according to the state information.

[0015] In one embodiment, the step of dynamically adjusting the intensity of the denoising process and the de-jitter process according to the state information includes:

[0016] If the speed exceeds a predetermined threshold, increasing the intensity of the denoising process and the de-jitter process;

[0017] If the speed is lower than a predetermined threshold, the strength of the denoising process and the de-jitter process is reduced.

[0018] In one embodiment, before the step of controlling the vehicle-mounted camera to obtain the image information of the driver according to the status information, the step further includes:

[0019] Adjust the shutter speed, aperture size and ISO sensitivity of the vehicle camera.

[0020] In one embodiment, the step of controlling the vehicle-mounted camera to obtain the image information of the driver according to the status information includes:

[0021] If the speed exceeds a predetermined threshold, increasing the shooting frequency of the vehicle-mounted camera;

[0022] If the speed is lower than a predetermined threshold, the shooting frequency of the vehicle-mounted camera is reduced.

[0023] In a second aspect, the present application provides a portrait processing device, the device comprising:

[0024] A state information detection unit, used to detect the state information of the current new energy vehicle;

[0025] an image information acquisition unit, configured to control the vehicle-mounted camera to acquire image information of the driver according to the status information; and

[0026] The determination unit is used to perform denoising on the image information and determine the driving state of the driver based on the denoised image information.

[0027] The above-mentioned portrait processing method, device, electronic device and medium provided by the present application have at least the following beneficial effects: 1. Intelligent detection: The present invention detects the state of the car before acquiring the image, and acquires the driver's photo according to the state of the car, which can more effectively avoid the image blur problem caused by the shaking of the car and improve the imaging quality. 2. Optimization processing: The present invention performs denoising and de-shaking processing on the acquired photos to further improve the imaging quality. Compared with the prior art, this processing method can not only improve the quality of the image, but also adjust the processing strategy according to the state of the car (such as speed, direction, etc.), so that the quality of the processed image is more stable. 3. Reduce costs: Compared with using hardware devices (such as stabilizers) to reduce the shaking of the car, the present invention processes the image through software algorithms without adding additional hardware devices, thereby reducing the cost of the car. 4. Improve user experience: Since the present invention can improve the imaging quality, it can provide a better driving experience, allowing drivers and passengers to obtain clearer and more stable images during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A schematic diagram of the structure of an electronic device provided in the first embodiment of the present application;

[0030] Figure 2 A flowchart of a portrait processing method provided in the second embodiment of the present application;

[0031] Figure 3 A schematic diagram of functional modules of a portrait processing device provided in the third embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0033] First embodiment:

[0034] Figure 1A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which can be used in the present application Figure 1 The schematic diagram shown is used to describe an electronic device 100 that is an example of an electronic device and a medium for implementing the portrait processing method, device, electronic device, and medium of an embodiment of the present application.

[0035] like Figure 1 The electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, and an output device 108. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structures of the electronic device 100 shown are only exemplary and not restrictive. The electronic device may have Figure 1 Some of the components shown may also have Figure 1 Other components and structures are not shown.

[0036] The processor 102 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0037] It should be understood that the processor 102 in the embodiment of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0038] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media.

[0039] It should be understood that the storage device 104 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0040] Among them, one or more computer program instructions can be stored on the computer-readable storage medium, and the processor 102 can execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present application described below and / or other desired functions. Various applications and various data, such as various data used and / or generated by the application, can also be stored in the computer-readable storage medium.

[0041] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0042] Second embodiment:

[0043] Reference Figure 2 The flowchart of a portrait processing method shown in FIG. 1 is applied to new energy vehicles and specifically includes the following steps:

[0044] Step S100, detecting the current status information of the new energy vehicle;

[0045] Specifically, before obtaining the driver's image information, the status of the new energy vehicle can be detected through the on-board sensors, including speed (obtained through the vehicle speed sensor) and direction (obtained through the steering wheel angle sensor) as well as the displacement stroke of the shock absorber in the vehicle suspension (obtained through the displacement sensor) and other information.

[0046] Based on this information, the system will automatically adjust the shooting parameters of subsequent on-board cameras, such as shutter speed (set to 1 / 50 second), aperture size (set to F2.8) and ISO sensitivity (set to 100) to ensure that clear photos of the driver are obtained.

[0047] Step S200, controlling the vehicle-mounted camera to obtain image information of the driver according to the status information;

[0048] Specifically, after obtaining the aforementioned status information, the driver's photo can be obtained using the vehicle-mounted camera. In this step, we use a high-definition camera with a resolution of 1280*720, which is installed inside the new energy vehicle and faces the driver's seat.

[0049] Step S300, denoising the image information, and judging the driving state of the driver based on the denoised image information.

[0050] In one embodiment, before the step of performing denoising on the image information, the method further comprises:

[0051] De-jittering is performed on the image information.

[0052] Specifically, after obtaining the driver's photo, firstly, de-shaking is performed to eliminate the image blur caused by the shaking of the car. In this step, we use a de-shaking algorithm based on motion estimation and compensation, which can effectively eliminate the motion blur in the image; then, denoising is performed to remove the noise interference in the image. In this step, a denoising algorithm based on wavelet transform can be used, which can effectively remove Gaussian noise in the image.

[0053] In one embodiment, it also includes:

[0054] The strength of the denoising process and the de-jitter process is dynamically adjusted according to the state information.

[0055] Further, the step of dynamically adjusting the intensity of the denoising process and the de-jitter process according to the state information includes:

[0056] If the speed exceeds a predetermined threshold, increasing the intensity of the denoising process and the de-jitter process;

[0057] If the speed is lower than a predetermined threshold, the strength of the denoising process and the de-jitter process is reduced.

[0058] Specifically, this embodiment can be called an adaptive processing strategy. That is, the processing strategy is automatically adjusted according to the state of the new energy vehicle. For example, when the speed of the new energy vehicle is faster (such as the speed exceeds 60km / h), the system may increase the intensity of denoising and de-jittering (such as setting the threshold of the denoising algorithm to 0.05 and the motion estimation accuracy of the de-jittering algorithm to 1 / 4 pixel) to ensure image quality; and when the speed of the car is slower (such as the speed is lower than 30km / h), the system may reduce the processing intensity (such as setting the threshold of the denoising algorithm to 0.02 and the motion estimation accuracy of the de-jittering algorithm to 1 / 8 pixel) to save processing time and resources.

[0059] In one embodiment, the step of controlling the vehicle-mounted camera to obtain the image information of the driver according to the status information includes:

[0060] If the speed exceeds a predetermined threshold, increasing the shooting frequency of the vehicle-mounted camera;

[0061] If the speed is lower than a predetermined threshold, the shooting frequency of the vehicle-mounted camera is reduced.

[0062] Specifically, this embodiment mainly automatically adjusts the shooting strategy according to the speed of the new energy vehicle. For example, when the speed of the new energy vehicle exceeds a predetermined threshold (such as a speed exceeding 60km / h), the shooting frequency of the on-board camera will be increased to compensate for the shooting distortion caused by the excessive speed to ensure the image quality; and when the speed of the vehicle is lower than the predetermined threshold (such as a speed lower than 30km / h), the shooting frequency of the on-board camera will be reduced to adapt to the shooting requirements at a lower speed, thereby saving processing time and resources.

[0063] Furthermore, in order to improve the processing effect of the software algorithm and make it unaffected by image complexity and algorithm efficiency, the present technical solution also adopts an image processing algorithm based on deep learning. In this embodiment, an image processing algorithm based on a convolutional neural network can be used, which can automatically learn the optimal processing parameters through a large amount of training data (such as 10,000 driver photos), thereby greatly improving the processing effect.

[0064] Furthermore, in order to improve the image quality without increasing the cost of the car, the present technical solution also adopts a hardware acceleration solution based on FPGA. In this embodiment, a hardware acceleration solution based on Xilinx Zynq-7000 series FPGA can be used, which can make full use of the parallel processing capability and high-speed data transmission capability of FPGA, thereby greatly improving the speed and efficiency of image processing.

[0065] In summary, the technical solution mainly adopts the following technical means: 1. Use the on-board camera to obtain the driver's photo: Before obtaining the image, the state of the car is first detected by the on-board sensor, including speed, direction, shock absorber displacement stroke and other information. According to this information, the shooting parameters are automatically adjusted to ensure that a clear driver's photo is obtained. 2. Image preprocessing: After obtaining the driver's photo, first perform de-shaking processing to eliminate the image blur caused by the shaking of the car. Then, de-noising is performed to remove noise interference in the image. These two steps can significantly improve the imaging quality. 3. De-noising adaptive processing strategy: Automatically adjust the processing strategy according to the state of the car. For example, when the car is fast, the intensity of de-noising and de-shaking can be increased to ensure image quality; when the car is slow, the intensity of de-noising and de-shaking can be reduced to save processing time and resources. 4. Shooting adaptive processing strategy: Automatically adjust the shooting strategy according to the state of the car. For example, when the speed of a new energy vehicle exceeds a predetermined threshold, the shooting frequency of the on-board camera will be increased to compensate for the shooting distortion caused by the excessive speed to ensure image quality; when the speed of the vehicle is lower than the predetermined threshold, the shooting frequency of the on-board camera will be reduced to adapt to the shooting requirements at a lower vehicle speed, thereby saving processing time and resources. 5. Optimize software algorithm: In order to improve the processing effect of the software algorithm and make it unaffected by image complexity and algorithm efficiency, this technical solution adopts an image processing algorithm based on deep learning. This algorithm can automatically learn the optimal processing parameters through a large amount of training data, thereby greatly improving the processing effect. 6. Hardware optimization: In order to improve the imaging quality without increasing the cost of the vehicle, this technical solution adopts a hardware acceleration solution based on FPGA. This solution can make full use of the parallel processing capability and high-speed data transmission capability of FPGA, thereby greatly improving the speed and efficiency of image processing.

[0066] Third embodiment:

[0067] See also Figure 3 A portrait processing device is shown, the device comprising:

[0068] A state information detection unit 10, used to detect the state information of the current new energy vehicle;

[0069] An image information acquisition unit 20, configured to control the vehicle-mounted camera to acquire image information of the driver according to the status information; and

[0070] The determination unit 30 is used to perform denoising on the image information and determine the driving state of the driver according to the denoised image information.

[0071] Furthermore, this embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processing device, the steps of any one of the portrait processing methods provided in the above-mentioned embodiment 2 are executed.

[0072] The computer program product of a portrait processing method, device and system provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments and will not be repeated here.

[0073] Due to the advanced nature of this technical solution, it can be widely used in application fields such as intelligent automobile driving, intelligent traffic management, and intelligent security systems. First, this technical solution can improve the portrait processing quality of vehicle-mounted cameras, which helps to improve the accuracy and reliability of driver monitoring, passenger monitoring, driver behavior analysis, etc., thereby improving driving safety and comfort, and meeting the demand for high-quality portrait processing technology in the field of intelligent automobile driving. Secondly, this technical solution can provide clearer and more stable images, which helps to improve the recognition and tracking capabilities of intelligent traffic management and intelligent security systems, and is of great significance to improving urban traffic management and public safety. Finally, this technical solution can adjust the processing strategy according to the state of the car, improve the flexibility and adaptability of the processing, and meet the needs of different application scenarios. Therefore, this technical solution has broad market prospects and application value.

[0074] It should be noted that the above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0075] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0076] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0077] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0078] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0080] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

Claims

1. A portrait processing method, applied to new energy vehicles, characterized in that: The method comprises: Detecting the current state information of the new energy vehicle; the state information includes the displacement stroke of the shock absorber in the vehicle suspension; Controlling the vehicle-mounted camera to obtain image information of the driver according to the status information; and Performing denoising and de-jittering processing on the image information, and dynamically adjusting the intensity of the denoising and de-jittering processing according to the displacement stroke of the shock absorber in the automobile suspension; The driving state of the driver is determined based on the image information after the denoising and de-shaking processing.

2. The method according to claim 1, characterized in that The state information also includes speed or direction.

3. The method according to claim 2, characterized in that Also includes: The strength of the denoising process and the de-jitter process is dynamically adjusted according to the state information.

4. The method according to claim 3, characterized in that The step of dynamically adjusting the intensity of the denoising process and the de-jitter process according to the state information comprises: If the speed exceeds a predetermined threshold, increasing the intensity of the denoising process and the de-jitter process; If the speed is lower than a predetermined threshold, the strength of the denoising process and the de-jitter process is reduced.

5. The method according to claim 1, characterized in that Before the step of controlling the vehicle-mounted camera to obtain the image information of the driver according to the state information, the method further includes: Adjust the shutter speed, aperture size and ISO sensitivity of the vehicle camera.

6. The method according to claim 2, characterized in that The step of controlling the vehicle-mounted camera to obtain the image information of the driver according to the status information comprises: If the speed exceeds a predetermined threshold, increasing the shooting frequency of the vehicle-mounted camera; If the speed is lower than a predetermined threshold, the shooting frequency of the vehicle-mounted camera is reduced.

7. A portrait processing device, applied to new energy vehicles, characterized in that: The device comprises: A state information detection unit, used to detect the current state information of the new energy vehicle; the state information includes the displacement stroke of the shock absorber in the vehicle suspension; an image information acquisition unit, configured to control the vehicle-mounted camera to acquire image information of the driver according to the status information; and A determination unit is used to perform denoising and de-jittering on the image information, and dynamically adjust the intensity of the denoising and de-jittering according to the displacement stroke of the shock absorber in the automobile suspension; and is also used to determine the driving state of the driver based on the image information after the denoising and de-jittering processing.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the portrait processing method according to any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processing device, the steps of the portrait processing method according to any one of claims 1 to 6 are executed.

Images