System, operating aid method and operating device for evaluating an image

By using deep neural network combined with SSIM loss function, the problem of rapid changes in brightness and contrast in camera images is solved, and high-quality image processing under different lighting conditions is achieved, suitable for environmental assessment.

CN110895807BActive Publication Date: 2025-02-28ROBERT BOSCH GMBH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN201910777506.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-08-22
Filing Date
2019-08-22
Publication Date
2025-02-28
Estimated Expiration
2039-08-22

AI Technical Summary

Technical Problem

When processing images captured by a camera, it is difficult to effectively deal with the problem of rapid changes in brightness and contrast in the scene, resulting in unstable image quality and affecting the accuracy of environmental assessment.

Method used

Deep neural networks (DNN) are used to combine structural similarity index (SSIM) as a loss function, and by training deep neural networks, they can generate high-quality images under different lighting conditions to adapt to changes in brightness and contrast.

Benefits of technology

It realizes that the image processing system can generate high-quality output images under rapid changes in brightness and contrast, which is suitable for high-quality environmental assessment and improves the stability and accuracy of image processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110895807B_ABST
    Figure CN110895807B_ABST
Patent Text Reader

Abstract

The invention relates to a system (100, S) for evaluating and / or matching images (i) recorded by means of a camera (10), wherein the system is based on or takes as a basis a deep neural network (20), the deep neural network (20) being trained (S1) or the deep neural network being subjected to training, the training (S1) of the deep neural network being carried out on the basis of a loss function as a metric or the training of the deep neural network being carried out on the basis of a loss function as a metric, the loss function being based on a structural similarity index (SSIM'), and the structural similarity index (SSIM') being determined during the training (S1) on the basis of or on the basis of: an input image (i) of at least one underlying camera (10), an output image (o) of the deep neural network, in particular an output image (o) of a matching method by means of a deep neural network, as a predetermined target image (t).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a system for evaluating and / or matching images captured by means of a camera (in particular in an operating assistance system of a working device or a vehicle), the present invention relates to an operating method and a control unit for a working device and in particular for a vehicle, the present invention also relates to such a working device and in particular to such a vehicle. Background Art

[0002] Operation assistance systems (especially driver assistance systems) for work equipment and especially for vehicles are becoming increasingly popular. In the context of autonomous operation and autonomous driving, measures for environmental detection and environmental assessment are very important in order to be able to plan and execute the required control operations. The corresponding quality of the images that the work equipment detects from the surroundings is decisive in this regard. In order to improve the detected initial images In order to ensure the quality of the image, image processing measures can be used so that the decision process is provided to the processed secondary image in the environmental assessment. However, in the image assessment, changes in the exposure characteristics of the scene, in particular rapid changes in brightness and contrast, can cause problems in automated vision and recognition. Summary of the invention

[0003] In contrast, the system according to the invention has the advantage that even rapid changes in the brightness and / or contrast of the scene can be taken into account in an improved manner. This is achieved here by the technical solution according to the invention in the following manner: in particular in an operating assistance system of a working device or a vehicle, a system for evaluating and / or matching images captured by means of a camera is proposed, in which a deep neural network is used, which is trained or is trained, the training of the deep neural network is carried out based on a loss function as a metric or the training of the deep neural network is carried out based on a loss function as a metric, the loss function is based on a structural similarity index, and in the training, the structural similarity index is determined based on the following or the structural similarity index is determined based on the following: at least one input image of the underlying camera, an output image of the deep neural network and in particular an output image of a matching method by the deep neural network, as a predetermined target image. By means of the training process according to the invention, the properties of the underlying deep neural network can be particularly well matched with respect to the improved properties in the case of rapidly changing lighting properties, so that in image processing applications, the initial image can be appropriately processed and in particular enhanced with respect to the lighting properties, so that the initial image can be used for high-quality environmental assessment.

[0004] The system according to the invention can be understood and implemented in a purely hardware-technical manner (e.g. as a device for evaluating and / or matching images). For example, an implementation as an ASIC is conceivable. Alternatively, a purely methodological implementation is conceivable: for example, a methodological implementation in combination with a computer implementation of the system according to the invention, preferably in combination with or as a methodological implementation of machine vision (also called computer vision methods). It is also conceivable to have combined systems or hybrid systems in which parts of the invention are implemented in hardware and / or software or methodological manner, or parts of the invention are implemented in hardware and / or software or methodological manner.

[0005] In an advantageous embodiment of the system according to the invention, it is particularly advantageous if, during training, the respective input image and the respective target image present the same scene or essentially the same scene with different exposure characteristics (especially with regard to brightness and / or contrast). In this way, the characteristics can be trained particularly well, especially under different lighting characteristics.

[0006] Alternatively or additionally, provision may be made here that the scene represented in the input image is illuminated at most poorly or at most with a lower illumination intensity and / or is displayed at most with a lower contrast than the scene represented in the target image.

[0007] The target image is preferably provided by the same camera as the input image, or the target image is preferably provided by the same camera as the input image.

[0008] A particularly high degree of flexibility in the training results if the corresponding target images are provided at a different time than the provision of the input images or vice versa.

[0009] Furthermore, according to another embodiment of the present invention, if during training, multiple input images corresponding to one or more target images are based on one or more scenes, a high degree of matching is obtained for the underlying deep neural network.

[0010] Can be combined with loss function SSIM 损失 The structural similarity index SSIM' is used (in particular according to the following equation (I)):

[0011] SSIM 损失 (i, o, t) = 1-SSIM′ (i, o, t) (I)

[0012] The loss function is used, in particular, as a function of the input image i, the output image o, the target image t and / or as a metric for training, or the loss function is used for training the underlying deep neural network.

[0013] Alternatively or additionally, it is also conceivable to use the structural similarity index SSIM′ in a product representation, in particular according to the following equation (II):

[0014] SSIM′(i,o,t)=l(o,t) α c(o, t) β s(i,o) γ (II)

[0015] Furthermore, it is conceivable, alternatively or additionally, to use a structural similarity index in combination with: a first component or factor 1 describing the brightness and / or illumination intensity between the output image o and the target image t, a second component or factor c describing the contrast between the output image o and the target image t and / or a third component or factor s describing the structural component between the input image i and the output image o.

[0016] Each of the components or factors may be weighted by means of hyperparameters α, β, γ that serve as exponents of the respective factors.

[0017] As the first factor l, a factor in the form of equation (2) can be used:

[0018]

[0019] As the second factor c, a factor in the form of equation (3) can be used:

[0020]

[0021] As the third factor s, a factor in the form of equation (4) can be used:

[0022]

[0023] Here,

[0024] μ x It can represent the average value of an image or part of an image x;

[0025] μ y It can represent the average value of an image or part of an image y;

[0026] σ x 2 It can represent the variance of an image or part of an image x;

[0027] σ y 2 It can represent the variance of an image or part of an image y;

[0028] σ xy 2It can represent the covariance of images or parts of images x and y;

[0029] c1 may represent a predetermined or to-be-determined first variable, which is used to stabilize the division in the case of a small denominator, and in particular, there exists: c1=(k1L) 2 ;

[0030] c2 may represent a predetermined or to-be-determined second variable, which is used to stabilize the division in the case of a small denominator, in particular: c2=(k2L) 2 ;

[0031] c3 may represent a third variable that is predetermined or to be predetermined, and the third variable is used to stabilize the division in the case of a small denominator, and in particular, there exists: c3=c2 / 2;

[0032] L can represent the dynamic range of pixel values ​​and has in particular the following values: 每像素的位数 -1;

[0033] k1 may represent a first parameter which is predetermined or to be predetermined, depending on the application, and which has, for example, a value of 0.01, and / or

[0034] k2 may represent a second parameter which is predetermined or to be predetermined depending on the application and which has, for example, a value of 0.03;

[0035] These parameters can also be defined and selected as in the conventional SSIM case.

[0036] According to another preferred embodiment of the system according to the present invention, during training, a first component or factor and a second component or factor between an output image of the deep neural network and a desired image or a target image are determined as the first image and the second image, and / or, a third component or factor between an input image and an output image of the deep neural network is determined as the first image and the second image.

[0037] Additionally or alternatively, when training the underlying deep neural network, the index SSIM′ and / or the loss function SSIM 损失 The optimization process in this area is used as the basis.

[0038] The invention also relates to an operating assistance method for a work device and in particular for a vehicle, which in the case of a vehicle can also be referred to as a driving assistance method. The core concept of the operating assistance method is to provide and use a system configured according to the invention.

[0039] The invention also provides a control unit for a working device and in particular for a vehicle.

[0040] The control unit is provided for starting and / or carrying out the operation assistance method according to the invention and is provided for use with the operation assistance method according to the invention and / or is provided for using a system designed according to the invention.

[0041] The present invention also provides a working device. Here, the working device can be a vehicle, in particular a motor vehicle, which is configured for autonomous or partially autonomous driving. Applications in robotics and similar fields are also conceivable.

[0042] The vehicle is provided to be controlled or used by or with the aid of the operating assistance method according to the invention, has a control unit designed according to the invention and / or a system designed according to the invention, or uses such a control unit or such a system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Embodiments of the present invention are described in detail with reference to the accompanying drawings.

[0044] Figure 1 A working device according to the invention is shown in the form of a vehicle in a schematic block diagram, which uses an embodiment of the system according to the invention;

[0045] Figure 2 A flow chart showing an embodiment of a system according to the invention designed as a method for evaluating images;

[0046] Figure 3 A further specific embodiment of the system according to the invention for evaluating images is shown in the form of a block diagram.

[0047] Figure 4 An input image, a target image and an output image processed by means of an embodiment of a system or method according to the present invention are shown in comparison. DETAILED DESCRIPTION

[0048] Next reference Figures 1 to 4 The embodiments and technical background of the present invention are described in detail. The same reference numerals are used to represent the same and equivalent elements and components with the same or equivalent effects. The mentioned elements and components are not described in detail again every time they appear.

[0049] The features shown and other characteristics may be isolated from one another and combined with one another in any desired form without departing from the core of the present invention.

[0050] Figure 1 A working device 1 designed according to the invention in the form of a vehicle 1 ′ is shown in a schematic block diagram, which uses an embodiment of a system 100 or a method S according to the invention.

[0051] A vehicle 1 ′ according to the invention is schematically shown with a body 2 , wheels 4 , a drive 30 with a drive train 31 and a system 40 for steering and braking by means of a steering and / or braking system 41 .

[0052] Furthermore, a control unit 50 is provided, for example in the form of a vehicle control unit or a motor control unit, which is connected via a control and / or detection line 51 to the drive 30 and to the system 40 for braking and steering.

[0053] Furthermore, the control unit 50 is also equipped with the cameras 10 (ie the second camera 10 - 2 and the first camera 10 - 1 ) via a control and / or detection line 51 in order to detect the surroundings of the vehicle 1 ′.

[0054] During operation of the vehicle 1', images of the surroundings of the vehicle 1' are provided to the control unit 50 via the camera 10 and to the image processing and image evaluation device using the deep neural network 20. According to the invention, the underlying deep neural network 20 is designed as part of the control unit 50 and thus as part of the system 100 for evaluating and / or processing images, so that the underlying deep neural network is therefore based on a training method S1 carried out according to the invention, which is described below and is part of the overall method S for evaluating and / or processing images.

[0055] Figure 2 A flow chart of one specific embodiment of a system 100 according to the invention, which is designed as a method S, is shown for evaluating and / or processing images.

[0056] The method S according to the present invention is divided into a training phase or training module S1 (also referred to as training for short) and an application phase or application module S2 (also referred to as application for short).

[0057] In step S1-1 of training S1, an input image i is provided by camera 10. Furthermore, a target image t is provided by a further step S1-2 (here also by means of camera 10), wherein the target image t shows the same or essentially the same scene as the input image i in the field of view of camera 10 - but has different exposure characteristics or lighting characteristics than the input image (in particular with regard to brightness and / or contrast).

[0058] Then, based on the working principle of the underlying deep neural network 20, an output image o is generated according to the determined parameterization. In subsequent steps S1-4, this output image can be evaluated and compared with respect to the input image i and the target image t, in order to achieve a learning adaptation of the parameterization of the underlying deep neural network 20, if necessary in conjunction with iterative training S1.

[0059] The actual training S1 is then followed by an application phase S2. The actual application S2 comprises a detection S2-1 of an image and an evaluation S2-2 of the detected image. The underlying working device 1 and its components can then be controlled S2-3 based on the result of the image evaluation S2-2.

[0060] It is thus conceivable, for example, to control or regulate the drive device 30 , the control device and the brake device 40 of the vehicle 1 ′ in conjunction with partially automated or fully automated driving based on the image evaluation S2 - 2 .

[0061] Figure 3 A further specific embodiment of a system 100 according to the invention for evaluating images is shown in the form of a block diagram.

[0062] Again in combination with the training S1 of the method S or system 100 according to the present invention, one or more input images i are provided in a first process S1-1 by one or more cameras 10, one or more target images t are provided in a second process S1-2, and in combination with the generation S1-3 of the output image o, the input images and target images are evaluated by the underlying deep neural network 20 in order to perform a corresponding learning process.

[0063] Figure 4 An input image i, a target image t and an output image processed by means of an embodiment of the system 100 or the method S according to the invention are shown in comparison.

[0064] It can be seen that: in combination with the training process S1, the neural network 20 is able to process the provided input image i based on the exposure / illumination characteristics appearing in the target image t, so that the output image o carries basically the same structural components as the input image i, but has corresponding illumination characteristics or exposure characteristics (especially in terms of brightness and contrast) due to the learning process S1.

[0065] These and other features and characteristics of the present invention are further elucidated with reference to the following statements:

[0066] The determination of the structural similarity index (also called SSIM) is a well-known image quality measure that is often used to determine the perceived quality of digital television and movie films.

[0067] According to the present invention, a new application of a metric as a loss function of a deep neural network system 20 or DNN system (DNN: deep neural network) is used for direct image illumination transformation and imaging.

[0068] For camera-based systems, determining the existing lighting conditions is an important issue. The general robustness of the methods or algorithms of different computer vision algorithms or machine vision techniques strongly depends on the external lighting characteristics. In different core or primary applications, failure to recognize or respond to sudden changes in lighting (e.g. when entering a tunnel) or failure to recognize or respond to long-term changes in natural lighting (e.g. at sunset) may lead to a drastic degradation of the camera input quality and also to a degradation of the computer vision function or machine vision function based on this input in the quality of its output.

[0069] Traditional image enhancement techniques or algorithms can only improve image quality in specific scenarios.

[0070] The object of the present invention is to provide computer vision methods or machine vision methods with high-quality images in any scenario. Machine learning methods or machine learning methods can avoid this complexity, but there are still problems here.

[0071] It is difficult to ensure an invariant image structure, i.e. to ensure that artifacts on the output image due to the machine learning process are avoided. This property is very advantageous - sometimes even essential - for safety-critical or safety-related automotive applications. The method proposed according to the present invention avoids this problem.

[0072] When using deep / deep neural networks (as part of a machine learning method), so-called loss functions can be defined, which are minimized during the training phase and whose value should therefore be minimal. The loss function defines a metric for the deep neural network during training. Lower values ​​of the loss function correspond to improved efficacy or performance of the neural network and of the method implemented with the aid of the neural network.

[0073] Considering the supervised learning problem for image modification, the corresponding loss function value is usually or often calculated between the modified image (output) and the desired image (target).

[0074] Here, the most commonly used loss function is based on the so-called L 2 The loss or squared loss L is defined by the following equation:

[0075]

[0076] Among them, y i is the target value and f(x i ) indicates the current output.

[0077] The problem with this approach is that the same object structure is required in both the input image and the target image, because otherwise the loss function will also detect this structural difference.

[0078] Since only the variations in lighting situations should be eliminated, input and target images from different positions must be used. However, the image structure (eg geometric content) will differ between any input and selected target images.

[0079] According to the invention, the Structural Similarity Index or a modified version of the SSIM index SSIM' is used (ie used as a metric) in order to find a solution to this problem.

[0080] The original Structural Similarity Index SSIM or SSIM Index is a widely used method mainly used to predict or generally describe the quality of digital videos and images.

[0081] The original rule for calculating the structural similarity index SSI (also referred to as SSIM index above and below) causes a comparison between a first image or a partial image (window) x and a second image or a partial image (window) y (of the same size / number of pixels, respectively). The calculation rule is described again in equation (1). The calculation rule is converted into three different parts l, c and s represented in product form by factorization. This product representation can be obtained again in the expression of equation (5) with the help of the definitions of equations (2) to (4).

[0082]

[0083]

[0084]

[0085]

[0086] SSIM(x,y)=l(x,y)·c(x,y)·s(x,y) (5)

[0087] The term l(x,y) describes the brightness between images or parts of images x, y of the same size. The term c(x,y) describes the contrast between the images or parts of images x, y. The term s(x,y) describes the structural parts or components between the images or parts of images x, y.

[0088] In particular, these terms describe differences or variations in respective variables (brightness / luminosity, contrast and structure) between the images or image parts x, y.

[0089] For example, when the window or section is moved along the entire image or when the window or section is moved along the entire image, the expression μ is calculated on a window or section of the image (especially smaller than the corresponding entire image) (for example on a partial image) x , μ y , The value of .

[0090] For equations (1) to (5), the various formula symbols have the following meanings:

[0091] μ x : the average value of x;

[0092] μ y : the mean value of y;

[0093] σ x 2 : variance of x;

[0094] σ y 2 : variance of y;

[0095] σ xy 2 : covariance of x and y;

[0096] c1: Variable used to stabilize the division in the case of small denominator, for example: c1 = (k1L) 2 ;

[0097] c2: Variable used to stabilize the division in the case of a small denominator, for example: c2 = (k2L) 2 ;

[0098] c3: a variable used to stabilize the division in the case of a small denominator, for example: c3 = c2 / 2;

[0099] L: Dynamic range of pixel values, which has the following values, for example: 2 每像素的位数 -1;

[0100] k1: Depending on the application, it has a value of 0.01, for example;

[0101] k2: Depending on the application, it has a value of 0.03, for example.

[0102] The basic idea of ​​the present invention is that in the training S1 of the underlying method S (for example, based on a deep neural network 20 (DNN)), the SSIM index is not analyzed and processed for direct comparison of two images, but a relationship between three images is established through a modified index SSIM′, namely, the input or input image i, the output or output image o of the underlying image processing process (for example, based on a deep neural network 20 (DNN)) and the target image t (also called the expected image or desired image), which relationship is usually recorded or recorded at a different time than the input image i and under different exposure characteristics of the imaged scene than the input image, but otherwise shows a similar scene, that is, a scene that is the same or substantially the same as the input image i.

[0103] (i) The term l(x, y) for brightness / light density and the term c(x, y) for contrast are determined between the output image (output) o and the target image (target) t—ie, in particular, the variables l(o, t) and c(o, t) are ascertained.

[0104] (ii) Determine the structural term s(x, y) between the input image (input) i and the output image (output) o - that is, determine the parameter s(i, o).

[0105] (iii) The SSIM index is then calculated overall based on the factorization according to equation (5), wherein, however, weighting is performed according to equation (5′) by means of hyperparameters α, β, γ in exponential form, which are predefined and / or determined according to the application. Thus, a modified SSIM index - SSIM′ - is obtained:

[0106] SSIM′(i,o,t)=l(o,t) α c(o, t) β s(i,o) γ (5′)

[0107] In this way, the underlying deep neural network 20 (DNN) used is forced to learn the lighting characteristics with brightness and contrast based on the target image t and obtain the structure from the input image i.

[0108] Since the modified SSIM index (SSIM′) is or provides a similarity measure having values ​​between 0 and 1, it is necessary to subtract the sought value of the SSIM index or measure from the value 1 in order to obtain the loss function (referred to as SSIM 损失 ).

[0109] Therefore, the loss function defined according to the present invention has a configuration according to the following equation (6):

[0110] SSIM损失 (i, o, t) = 1-SSIM′ (i, o, t) = 1-l (o, t) α c(o, t) β s(i,o) γ (6)

[0111] Again, the symbol i denotes an input image, o an output image and t a target image. The parameters α, β, γ - as already mentioned - are so-called hyperparameters which define the relative weights of the individual parts and which are predetermined or predetermine these hyperparameters - for example depending on the respective application, wherein in particular an optimization of the underlying image processing process can also be used to generate an output image o given an input image i and a given target image t, and in particular the training of the underlying deep neural network can be used.

[0112] In machine learning methods, hyperparameters are usually understood to be coefficients whose values ​​cannot be easily determined or specified and for which it is actually necessary to adapt or even optimize the parameters during the optimization process of the system. This can be done by setting the corresponding hyperparameters to fixed values ​​before the system is trained and by trying and adapting a number of these fixed configurations.

[0113] As a further modification, the parameter expression μ can be changed x , μ y , The kernel of the equation is the computational expression of these parameter expressions. It has been determined that the use of the Gaussian kernel is superior to the most commonly used primitive kernel.

Claims

1. A system (100, S) for evaluating and / or matching images (i) recorded by means of a camera (10), in which system: Using deep neural networks (20); The deep neural network (20) has been trained (S1), or the deep neural network has been trained; The training (S1) of the deep neural network (20) is performed based on a loss function as a metric, or the training of the deep neural network is performed based on a loss function as a metric; The loss function is based on the structural similarity index (SSIM'); In the training (S1), the structural similarity index (SSIM') is obtained based on the following or the structural similarity index is obtained based on the following: at least one input image (i) of the camera (10), an output image (o) of the deep neural network (20), a predetermined target image (t), The structural similarity index SSIM' is used in product expression according to the following equation (II): SSIM′(i,o,t)=l(o,t) α ·c(o,t) β ·s(i,o) γ (II) and / or using the structural similarity index in combination with: a first component or factor l describing the brightness between the output image o and the target image t, a second component or factor c describing the contrast between the output image o and the target image t, and a third component or factor s describing the structural component between the input image i and the output image o, wherein, Each of these components or factors is weighted by means of hyperparameters α, β, γ that act as exponents of the corresponding factors.

2. The system (100, S) according to claim 1, wherein In the training (S1), the corresponding input image (i) and the corresponding target image (t) present the same scene or substantially the same scene, and the same scene or substantially the same scene has different exposure characteristics; The scene represented in the input image (i) is at most poorly illuminated or at most with a lower illumination intensity and / or is displayed at most with a lower contrast than the scene represented in the target image (t); The target image (t) is provided by the same camera as the input image (i), or the target image is provided by the same camera as the input image; and / or The target image (t) is provided at a different time from that at which the input image (i) is provided, or the target image is provided at a different time from that at which the input image is provided.

3. A system (100, S) according to claim 1 or 2, in which system, in the training (S1), multiple input images (i) corresponding to one or more target images (t) are based on one or more scenes.

4. The system (100) according to claim 1 or 2, wherein the loss function SSIM is combined with the loss function SSIM. 损失 The structural similarity index SSIM' is used according to the following equation (I): SSIM 损失 (i,o,t)=1-SSIM′(i,o,t) (I) The loss function is a function of the input image i, the output image o and the target image t and / or a measure of the training (S1) of the underlying deep neural network (20).

5. The system (100) according to claim 1 or 2, wherein Using the term according to the form of equation (2) as the first component or factor l; Using a term according to the form of equation (3) as the second component or factor c; and / or, using a term according to the form of equation (4) as a third component or factor s; and / or, x represents the first image or a portion of the image; y represents the second image or a portion of the image; μ x represents the average value of the first image or the partial image x; μ y represents the average value of the second image or the partial image y; σ x 2 represents the variance of the first image or the partial image x; σ y 2 represents the variance of the second image or the partial image y; σ xy 2 represents the covariance of the first image and the second image or the partial images x and y; c1 represents a predetermined or to-be-determined first variable, which is used to stabilize the division in the case of a small denominator, and there exists: c1=(k1L) 2 ; c2 represents a predetermined or to-be-determined second variable, which is used to stabilize the division in the case of a small denominator, and there exists: c2=(k2L) 2 ; c3 represents a third variable that is predetermined or to be predetermined, and the third variable is used to stabilize the division in the case of a small denominator, and there exists: c3=c2 / 2; L represents the dynamic range of the pixel value, and L has a value of 2 每像素的位数 -1; k1 denotes a first parameter which is predetermined or to be predetermined depending on the application, k1 having a value of 0.01; and / or k2 denotes a second parameter which is predetermined or to be predetermined depending on the application and has a value of 0.

03.

6. A system (100) according to claim 5, in which, in the training (S1), a first component or factor (l) and a second component or factor c between the output image (o) of the deep neural network (20) and the desired image / target image (t) are determined as the first image and the second image (x, y), and / or a third component or factor between the input image (i) and the output image (o) of the deep neural network (20) is determined as the first image and the second image (x, y).

7. The system (100) according to claim 1 or 2, wherein in the training (S1) of the deep neural network (20) based on the index (SSIM') and / or the loss function (SSIM 损失 ) as the basis for the optimization process.

8. The system (100) according to claim 1, which is a system in an operation assistance system of a working device (1) or a vehicle (1').

9. The system (100) according to claim 1, wherein the output image (o) of the deep neural network (20) is an output image of a matching method through the deep neural network (20).

10. The system (100) according to claim 2, wherein the different exposure characteristics are exposure characteristics in terms of brightness and / or contrast.

11. An operation assistance method for a working device (1), in which the system (100, S) according to any one of claims 1 to 10 is used. 12 . The operating assistance method according to claim 11 , for a vehicle ( 1 ′).

13. A control unit (50) for a working device (1), the control unit being configured to start and / or implement an operation assistance method according to claim 11 or 12, the control unit being configured to be used with the aid of the operation assistance method according to claim 11 or 12, and / or to use a system (100, S) according to any one of claims 1 to 10 in the control unit.

14. A control unit (50) according to claim 13, for use in a vehicle (1').

15. A working device (1), which is configured to be controlled or used by or with the aid of an operation assistance method according to claim 11 or 12, the working device having a control unit (50) according to claim 13 or 14 and / or the working device having or using a system (100, S) according to any one of claims 1 to 10.

16. The working device (1) according to claim 15, comprising a vehicle (1').

Citation Information

Patent Citations

  • Image domain conversion network based on generative adversarial networks (GAN) and conversion method

    CN108171320A