Training methods and equipment for image signal processing environments suitable for machine vision

Abstract

This invention relates to a training method, apparatus, and readable medium for an image signal processing environment. The computing device of this invention includes a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the training method for the image signal processing environment. This invention can improve the semantic quality of processed images.

Description

Technical Field

[0001] This invention generally relates to the field of neural networks. More specifically, this invention relates to training methods, image signal processing apparatuses, computing devices, computer-readable storage media, and computer program products suitable for image signal processing environments applicable to machine vision. Background Technology

[0002] In image signal processing (ISP) technology, tone mapping (TM) modules or dynamic range compression (DRC) modules, which aim to realistically reproduce scenes or please the human eye, have long been the core functional modules of image signal processing. This is because, limited by the display bit width of display devices, the high bit width data acquired by image acquisition sensors cannot be directly displayed on the display device and must be processed by the TM or DRC modules.

[0003] With the development of machine vision, more and more products no longer need to display images, but instead directly receive image data and make automatic decisions based on those images, such as robots, autonomous vehicles in industrial parks, and intelligent monitoring systems. These products typically include a core neural network to analyze the received image signals and make decisions based on the semantic information of the images. These products are also limited by bit width and require TM or DRC modules to process the image data. To enable machines to make better decisions based on the processed images, image processing technology needs to generate images suitable for machine interpretation; that is, the generated images should allow machines to extract high-quality semantic information as a basis for decision-making. Because machines perceive image semantic information differently than human eyes, the techniques for processing image data to generate machine-readable images differ from those for generating images suitable for human vision. To generate machine-readable images, an image signal processing technique suitable for machine vision is needed, particularly tone mapping or dynamic range compression techniques. Summary of the Invention

[0004] In order to at least partially solve the technical problems mentioned in the background art, the present invention provides a training method, an image signal processing device, a computing device, a computer-readable storage medium, and a computer program product suitable for an image signal processing environment of machine vision.

[0005] In one aspect, the present invention discloses a training method for an image signal processing environment applicable to machine vision, the image signal processing environment including a tone mapping framework. The training method includes: establishing a coach network; and training network layers in the tone mapping framework based on the weights of the coach network to obtain post-coached weights for the network layers; wherein the network layers generate half-saturation values ​​of pixel brightness based on the post-coached weights.

[0006] In another aspect, the present invention discloses an image signal processing apparatus suitable for machine vision, comprising: a tone mapping module for adjusting pixel brightness, which includes a network module trained based on the weights of a coach network to obtain post-coached weights, and generating a half-saturation value of pixel brightness based on the post-coached weights.

[0007] In another aspect, the present invention discloses a computer device including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the method described above.

[0008] In another aspect, the present invention discloses a computer-readable storage medium having stored thereon computer program code of a training method for an image signal processing environment suitable for machine vision, which, when run by a processing device, executes the method as described above.

[0009] In another aspect, the present invention discloses a computer program product comprising a computer program for a training method applicable to an image signal processing environment for machine vision, characterized in that the computer program, when executed by a processor, implements the steps of the method as described above.

[0010] In training an image signal processing environment suitable for machine vision, this invention introduces a coach network to train the network layers of the tone mapping framework in the image signal processing environment. This enables the trained network layers to output pixel brightness half-saturation values ​​that can be used in image signal processing suitable for machine vision based on the image to be processed, thereby realizing an image signal processing technology suitable for machine vision. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts. Wherein:

[0012] Figure 1 This is a schematic diagram illustrating the structure of the tone mapping framework according to an embodiment of the present invention;

[0013] Figure 2This is a schematic diagram illustrating the structure of an image signal processing environment / apparatus according to an embodiment of the present invention;

[0014] Figure 3 This is an example flowchart illustrating a training method for an image signal processing environment applicable to machine vision, according to an embodiment of the present invention.

[0015] Figure 4 This is an example flowchart illustrating a training method for an image signal processing environment applicable to machine vision, according to another embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram illustrating the structure of a training environment based on a coach network training network layer according to an embodiment of the present invention;

[0017] Figure 6 This is a schematic block diagram illustrating the components of a computing device according to an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0020] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0021] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0022] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] As mentioned earlier, some products, such as robots, now require decision-making based on semantic information in images. However, existing image signal processing techniques often produce images that are pleasing to the human eye or mimic human vision, neglecting semantic quality and thus being unsuitable for machine vision. Therefore, this invention provides an image signal processing technique suitable for machine vision, which uses a tone mapping framework to adjust pixel brightness to generate images suitable for machine interpretation.

[0024] Figure 1 A schematic diagram of the tone mapping framework according to an embodiment of the present invention is shown. Figure 1 As shown, this example tone mapping framework 100 includes a network module 101, a human eye compression module 102, a color-to-luminance module (RGB2L module) 103, and a luminance-to-color module (L2RGB module) 104. The tone mapping framework 100 is based on an improved human eye compression model. The network module 101 includes one network layer. The human eye compression module 102, based on the human eye compression model, generates an adjusted output pixel luminance according to the input pixel luminance, statistical information, and the half-saturation value σ (i.e., background information) of the image to be processed. The statistical information is global statistical information, which can also be expanded and combined with the image to be processed as input to the tone mapping framework 100. In one example, the human eye compression module 102 generates the output pixel luminance value based on the following formula:

[0025]

[0026] Where: L out L represents the output pixel brightness value output by the human eye compression module 102. in The input pixel brightness of the image to be processed is represented by the color-to-brightness module 103, which converts the pixel color values ​​of the input image into pixel brightness values. σ represents the half-saturation value of the pixel brightness, and n represents the sensitivity of the human eye, typically set to 1. The half-saturation value σ of the pixel brightness directly affects the local dynamic range compression effect. In existing technologies, σ is a complex function of global statistical information and local image content. In existing tone mapping frameworks for generating images suitable for human eye reception, the σ value is generally calculated using a calibrated formula. However, the σ value calculated in this way is not suitable for machine vision, and it is impossible to generate images suitable for machine interpretation using this σ value.

[0027] The tone mapping framework in this embodiment needs to perform dynamic range compression for machine vision. It requires adjustments based on image semantic information to accommodate local adaptation. Therefore, it needs to calculate a half-saturation value σ suitable for machine interpretation, ensuring that the output pixel brightness generated using this σ value is suitable for machine vision. In this embodiment, a neural network (i.e., Figure 1 The network layer of the network module 101 in the middle determines the σ value so that the human eye compression module 102 generates output pixel brightness suitable for machine vision based on the σ value.

[0028] The network layers can be multi-layer two-dimensional convolutional networks with a stride of 1, and can be a ResNet structure. After training, the network layers are used to generate half-saturation values ​​of pixel brightness based on the input image to be processed, i.e.:

[0029] σ=F(L in )

[0030] Where F is the local half-saturation feature extracted by the network layer from the image to be processed, and the output is the half-saturation value that completes local adaptation; σ value refers to the half-saturation value of all pixels in the image to be processed, and each pixel corresponds to a value.

[0031] The image to be processed also undergoes color-to-luminance conversion module 103, which converts the original RGB color space input image into a Lab color space image, that is, converts the input pixel color values ​​into input pixel luminance values. Human eye compression module 102 receives the Lab color space image and calculates the output pixel luminance L based on the half-saturation value from network module 101. out The brightness-to-color module 104 converts the output pixel brightness value into the output pixel color value in the RGB color space to generate the processed image.

[0032] In machine vision applications, the image data processed by the tone mapping framework 100 is typically used by the application device to perform certain tasks, such as decision-making. In many cases, the task processing unit of the application device is a neural network, and the image data processed by the tone mapping framework 100 is used as the input to this neural network. The tone mapping framework 100 and the neural network together constitute an image signal processing environment suitable for machine vision, which can also be called an image signal processing device. Figure 2 A schematic diagram of the image signal processing environment / apparatus of this embodiment is shown. Figure 2As shown, the image signal processing environment / apparatus includes a tone mapping framework (also referred to as a tone mapping module) 100 and a target network module 105. The tone mapping framework 100 is connected to the target network module 105, which is a neural network. The processed image output by the tone mapping framework 100 serves as the input to the target network module 105. The target network module 105 includes a target network that performs a specific task based on the processed image and outputs the task processing result, such as performing tasks like image recognition, classification, and detection. Alternatively, the tone mapping framework 100 and the target network together perform inference tasks based on the input image to be processed, completing a specific machine vision-based task.

[0033] Both the network layers and the target network of the tone mapping framework 100 need to be trained before they can be used. How to train the network layers and the target network to obtain an image signal processing environment suitable for machine vision is the key to achieving machine vision-friendly image signal processing technology in this embodiment. The following description of the training method for the machine vision-friendly image signal processing environment provided in this embodiment illustrates how to train the network layers and the target network.

[0034] Figure 3 An example flowchart illustrating a training method for an image signal processing environment applicable to machine vision, according to an embodiment of the present invention, is shown. In this example method, a coach network is introduced during training, and network layers are trained based on the trained coach network. This ensures that the network layers in the tone mapping framework 100 are trained before being connected to the target network module 105 for joint training, thus avoiding convergence failure or poor generalization ability during joint training with the target network. Figure 5 A schematic diagram of the structure of a training environment based on a coach network for training network layers, according to an embodiment of the present invention, is shown. Figure 5 As shown, tone mapping framework 100 is connected to coach network module 106, and the output of tone mapping framework 100 serves as the input of coach network module 106. The coach network in coach network module 106 completes task processing based on the output of tone mapping framework 100 and outputs the task processing results.

[0035] like Figure 3 As shown, the training method includes:

[0036] In step 301, a coach network is established. For example, the coach network can be a backbone network plus a header network, where the backbone network can adopt a ResNet structure (e.g., a multi-layer ResNet structure), and the header network can be several layers (e.g., 3 layers, 4 layers, etc.) of fully connected layers plus a softmax structure. The backbone network is used for feature extraction, and the header network is used for classification.

[0037] Alternatively, a large-scale open-source dataset (such as those in ImageNet, COCO, or VOC databases) can be used to train the coach network to determine its weights. Alternatively, a pre-trained coach network from an open-source database can be used, omitting the step of training the coach network.

[0038] In step 302, the network layers in the tone mapping framework are trained based on the weights of the coach network to obtain the post-coached weights of the network layers. In this step, the tone mapping framework 100 is... Figure 5 Then, connecting to the coach network module 106 established in step 301, locking the weights of the coach network, and training the network layers in the tone mapping framework 100. This training can be a self-supervised learning process, enabling the network layers to learn general feature representation capabilities from a large amount of unlabeled data. For example, this training can be performed based on the gradient backpropagation of the loss function of the coach network.

[0039] The trained network layers can obtain post-training weights and generate half-saturation values ​​for pixel brightness based on these weights for the input image to be processed. The tone mapping framework 100 then uses these half-saturation values ​​to adjust the pixel brightness of the image to be processed, generating a processed image suitable for machine vision. These half-saturation values ​​improve semantic extraction (image recognition) capabilities and enhance the performance of the backend network.

[0040] The image signal processing environment training method of this invention introduces a coach network from an open-source database when training network layers. The network layers are trained based on this pre-trained coach network, enabling the acquisition of network layer weights and the output of pixel brightness half-saturation values ​​suitable for machine vision based on the input image to be processed. This invention utilizes a data-driven mechanism to learn the half-saturation values ​​that influence local information in the human eye compression model, thereby modifying the human eye compression model to suit machine vision and enabling semantic quality optimization based on local image content. Furthermore, this hybrid architecture of the tone mapping framework in this invention also effectively ensures that the output image is human-readable.

[0041] Figure 4 A flowchart illustrating a training method for an image signal processing environment applicable to machine vision, according to another embodiment of the present invention, is shown. In this embodiment, the method includes, in addition to... Figure 3The steps included in the embodiment further include a step of initial training the network layer using data based on human eye calibration, and a step of connecting the tone mapping framework 100 to the target network module 105 after determining the weights of the network layer to fine-tune the target network and obtain the fine-tuned weights of the target network. The image signal processing environment, tone mapping framework, and network layer composition in this embodiment are all similar to... Figure 3 The embodiments are the same and can be referred to accordingly. Figure 1-3 The description will not be repeated here.

[0042] like Figure 4 As shown, this example method includes:

[0043] In step 401, the network layers in the tone mapping framework are trained using a human eye training dataset. The significance of adding this step in this embodiment is to initially establish the initial weights of the network layers before training with the coach network, reducing the difficulty of subsequent training. The human eye training dataset used in this step is a dataset generated through human eye calibration, not a machine vision calibration dataset. That is, such a human eye training dataset is based on existing image signal processing techniques, and the associated processed images are suitable for human eye reception, not machine interpretation. However, since this invention is based on modifications to the original tone mapping framework and the original human eye compression model, although the human eye-calibrated dataset cannot establish weights for the network layers suitable for machine image interpretation, using it for training can establish initial weights. Further adjustments based on these initial weights can then yield the final weights.

[0044] Optionally, this step may include: generating a human eye training dataset through human eye calibration; and importing the human eye training dataset into the tone mapping framework 100 to supervise the training of the network layer and generate the initial weights of the network layer. For example, during training, the unprocessed image is used as the input to the tone mapping framework 100, and the human eye-calibrated image, generated based on the original tone mapping framework and the original human eye compression model and suitable for human eye reception, is used as the output of the tone mapping framework 100 to supervise the training of the network layer.

[0045] In step 402, a coaching network is established. This step is the same as step 301 described above, and will not be repeated here.

[0046] In step 403, the network layers are trained based on the coach network. The difference between this step and step 302 is that the initial weights of the network layers have already been established in step 401. The training in this step is an adjustment based on the initial weights, so that the weights based on machine vision can converge faster. Other details are the same as in step 302, and will not be repeated here.

[0047] This step establishes the weights of the network layers, generating a trained network module 101 that can produce pixel brightness half-saturation values ​​that meet the requirements of machine vision.

[0048] In step 404, the target network is trained based on the post-coach weights. This step is for fine-tuning the target network, and the training environment can be found in [reference needed]. Figure 2 In this step, the tone mapping framework 100 trained in step 403 is disconnected from the coach network module 106, but connected to the separately trained target network module 105. The output of the tone mapping framework 100 is used as the input of the target network module 105. At this point, the target network module 105 has been trained separately, but it is not connected to the tone mapping framework 100 for joint training.

[0049] In this step, you can either not lock any weights, including training the network layers and the target network together, or lock only one set of weights and train the other set of weights. For example, you can lock the weights of the network layers and fine-tune the weights of the target network during training. Based on an open-source image database, you can obtain the fine-tuned weights of the network layers and / or the target network through self-supervised training, so that the entire image signal processing environment can achieve the task accuracy after semantic quality improvement.

[0050] In other embodiments, the training method may include only steps 401 to 403; or only steps 402 to 404.

[0051] This invention also provides a computer-readable storage medium storing computer program code for a training method for an image signal processing environment applicable to machine vision. When the computer program code is run by a processing device, the training method for the image signal processing environment described above is implemented.

[0052] This invention also provides a computer program product, which includes a computer program for a training method for an image signal processing environment applicable to machine vision. When the computer program is executed by a processor, it implements the training method for the image signal processing environment as described above.

[0053] This invention also provides a computing device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the training method for the image signal processing environment as described above. Figure 6 A schematic block diagram of such a computing device is shown. Figure 6As shown, the computing device 6 in this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the various method embodiments described above. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the various device embodiments described above.

[0054] The computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the embodiments of the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 62 in the computing device 6.

[0055] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0056] The memory 61 can be an internal storage unit of the computing device 6, such as a hard disk or RAM of the computing device 6. The memory 61 can also be an external storage device of the computing device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computing device 6. Furthermore, the memory 61 can include both internal storage units and external storage devices of the computing device 6. The memory 61 is used to store the computer program and other programs and data required by the server. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0058] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0059] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0060] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0062] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0063] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0064] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0065] The foregoing can be better understood in accordance with the following terms:

[0066] Clause A1. A training method for an image signal processing environment applicable to machine vision, the image signal processing environment including a tone mapping framework, the training method comprising:

[0067] Establish a coaching network; and

[0068] Train the network layers in the tone mapping framework based on the weights of the coach network to obtain the coached weights of the network layers;

[0069] The network layer generates the half-saturation value of pixel brightness based on the weights after coaching.

[0070] Clause A2, according to the training method described in Clause A1, the tone mapping framework includes a human eye compression model, and the training method further includes:

[0071] Generate a human eye training dataset through human eye calibration; and

[0072] The human eye training dataset is imported into the tone mapping framework to supervise the training of the network layer and generate the initial weights of the network layer.

[0073] The step of training the network layer based on the weights of the coach network is to expand the training based on the initial weights.

[0074] Clause A3, the training method according to Clause A1, wherein the coaching network comprises:

[0075] The backbone network is a ResNet structure; and

[0076] The header network consists of several fully connected layers with a Softmax structure.

[0077] Clause A4, the training method according to Clause A3, wherein the training data of the coach network is one of ImageNet, COCO and VOC databases.

[0078] Clause A5, the training method according to Clause A3, wherein the step of training the network layer based on the weights of the coach network is self-supervised training.

[0079] Clause A6, the training method according to Clause A3, wherein the step of training the network layer based on the weights of the coach network is training by backpropagation of the gradient of the loss function of the coach network.

[0080] Clause A7, the training method according to Clause A1, wherein the network layer is a ResNet structure.

[0081] Clause A8, the training method according to Clause A1, wherein the network layer is a multi-layer two-dimensional convolutional network with a stride of 1.

[0082] Clause A9. The training method according to Clause A1, wherein the image signal processing environment includes a target network, and the training method further includes:

[0083] Based on the coached weights, the target network is trained to obtain the fine-tuned weights of the target network.

[0084] The image signal processing environment performs inference tasks based on the coached weights and the fine-tuned weights.

[0085] Clause A10, the training method according to Clause A2, wherein the human eye compression model generates output pixel brightness values ​​based on the following formula:

[0086]

[0087] Among them, L out L is the output pixel brightness value. in σ is the input pixel brightness value, and σ is the half-saturation value.

[0088] Clause A11, the training method according to Clause A10, wherein the tone mapping framework further includes:

[0089] A color-to-luminance module is used to convert the color value of an input pixel into the luminance value of the input pixel; and

[0090] A brightness-to-color conversion module is used to convert the output pixel brightness value into the output pixel color value.

[0091] Clause A12. A computer-readable storage medium having stored thereon computer program code of a training method for an image signal processing environment suitable for machine vision, which, when run by a processing device, performs the method described in any one of Clauses A1 to A11.

[0092] Clause A13. A computer program product comprising a computer program for a training method applicable to an image signal processing environment for machine vision, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described in any one of Clauses A1 to A11.

[0093] Clause A14. A computer apparatus comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the method described in any one of Clauses A1 to A11.

[0094] Clause A15, an image signal processing apparatus suitable for machine vision, comprising:

[0095] The tone mapping module, used to adjust pixel brightness, includes:

[0096] The network module is trained based on the weights of the coach network to obtain post-coach weights, and generates the half-saturation value of the pixel brightness based on the post-coach weights.

[0097] Clause A16, the image signal processing apparatus according to Clause A15, wherein the tone mapping module further comprises:

[0098] A human eye compression module is used to generate a human eye training dataset to train the network module and generate the initial weights of the network module.

[0099] The network module is trained based on the initial weights to obtain the post-coach weights.

[0100] Clause A17. The image signal processing apparatus according to Clause A15 further includes:

[0101] A target network module is used to train based on the coached weights to obtain the fine-tuned weights of the target network module.

[0102] The image signal processing device performs inference tasks based on the coached weights and the fine-tuned weights.

[0103] Clause A18. The image signal processing apparatus according to Clause A16, wherein the human eye compression module generates output pixel brightness values ​​based on the following formula:

[0104]

[0105] Among them, L out To output pixel brightness values, L in σ is the input pixel brightness value, and σ is the half-saturation value.

[0106] Clause A19, the image signal processing apparatus according to Clause A18, wherein the tone mapping module further comprises:

[0107] A color-to-luminance module, used to convert the color value of an input pixel into the luminance value of the input pixel; and

[0108] A brightness-to-color conversion module is used to convert the output pixel brightness value into the output pixel color value.

[0109] The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A training method for an image signal processing environment suitable for machine vision, the image signal processing environment including a tone mapping framework, the tone mapping framework including network layers and a human eye compression model, the training method comprising: Establish a coaching network, which includes a backbone network and a header network; as well as Train the network layers in the tone mapping framework based on the weights of the coach network to obtain the coached weights of the network layers; The network layer generates the half-saturation value of pixel brightness based on the weights after coaching. The half-saturation value of the pixel brightness is used to generate an image suitable for machine interpretation; the human eye compression model generates an output pixel brightness suitable for machine vision based on the half-saturation value of the pixel brightness.

2. The training method according to claim 1, further comprising: A human eye training dataset is generated through human eye calibration. as well as The human eye training dataset is imported into the tone mapping framework to supervise the training of the network layer and generate the initial weights of the network layer. The step of training the network layer based on the weights of the coach network is to expand the training based on the initial weights.

3. The training method according to claim 1, wherein... The backbone network is a ResNet structure; and The head network consists of several fully connected layers plus a Softmax structure.

4. The training method according to claim 3, wherein the training data of the coach network is one of ImageNet, COCO and VOC databases.

5. The training method according to claim 3, wherein the step of training the network layer based on the weights of the coach network is self-supervised training.

6. The training method according to claim 3, wherein the step of training the network layer based on the weights of the coach network is to perform training by backpropagation according to the gradient of the loss function of the coach network.

7. The training method according to claim 1, wherein the network layer is a ResNet structure.

8. The training method according to claim 1, wherein the network layer is a multi-layer two-dimensional convolutional network with a stride of 1.

9. The training method according to claim 1, wherein the image signal processing environment includes a target network, and the training method further includes: Based on the coached weights, the target network is trained to obtain the fine-tuned weights of the target network. The image signal processing environment performs inference tasks based on the coached weights and the fine-tuned weights.

10. The training method according to claim 2, wherein the human eye compression model generates output pixel brightness values ​​based on the following formula: in, The output pixel brightness value, For input pixel brightness values, The value is the half-saturation value.

11. The training method of claim 10, wherein the tone mapping framework further comprises: A color-to-brightness module is used to convert the color value of the input pixel into the brightness value of the input pixel; as well as A brightness-to-color conversion module is used to convert the output pixel brightness value into the output pixel color value.

12. A computer-readable storage medium storing computer program code of a training method for an image signal processing environment suitable for machine vision, wherein when the computer program code is run by a processing device, it performs the method according to any one of claims 1 to 11.

13. A computer program product comprising a computer program for a training method applicable to an image signal processing environment for machine vision, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 11.

15. An image signal processing apparatus suitable for machine vision, comprising: The tone mapping module, used to adjust pixel brightness, includes: The network module is trained based on the weights of the coach network to obtain post-trained weights, and generates half-saturation values ​​of the pixel brightness based on the post-trained weights. These half-saturation values ​​are used to generate an image suitable for machine interpretation. The coach network includes a backbone network and a header network. The human eye compression module is used to generate output pixel brightness suitable for machine vision based on the half-saturation value of the pixel brightness.

16. The image signal processing apparatus according to claim 15, wherein the initial weights of the network module are generated based on supervised training of a human eye-calibrated training dataset; and the network module is trained based on the initial weights to obtain the coached weights.

17. The image signal processing apparatus according to claim 15, further comprising: A target network module is used to train based on the coached weights to obtain the fine-tuned weights of the target network module. The image signal processing device performs inference tasks based on the coached weights and the fine-tuned weights.

18. The image signal processing apparatus according to claim 16, wherein the human eye compression module generates the output pixel brightness value based on the following formula: in, To output pixel brightness values, For input pixel brightness values, The value is the half-saturation value.

19. The image signal processing apparatus according to claim 18, wherein the tone mapping module further comprises: A color-to-brightness module is used to convert the color value of the input pixel into the brightness value of the input pixel; as well as The brightness-to-color module is used to convert the output pixel brightness value into the output pixel color value.

Images