A visual improvement method for the light contrast inside and outside the cabin of a large ship unloading engine room

Abstract

The present invention provides a visual enhancement method for the light contrast inside and outside the cabin of a large ship unloader, belonging to the field of port intelligent technology. The present invention introduces instance segmentation technology to accurately identify the hatch area, and combines the adaptive gamma value calculation method to dynamically optimize the contrast and brightness of the image in the cabin, and adds corresponding post-processing to solve the possible image distortion problem, so as to fundamentally solve the problem of strong light and dark contrast between the deck and the cabin during the operation of the ship unloader, and improve the safety of the grab ship unloader operation, the efficiency of the operation and the comfort of the operator.

Description

Technical Field

[0001] The present invention relates to the field of port intelligent technology, and in particular to a method, electronic equipment, storage medium and computer program product for visual enhancement of light contrast inside and outside a large ship unloading cabin. Background Art

[0002] In modern port logistics, grab ship unloaders are key cargo loading and unloading equipment. However, the extreme light contrast in the operating environment, especially the coexistence of dim light inside the cargo ship hatch and strong light outside the deck, seriously hinders the operator's accurate judgment of the situation inside the cabin, thereby increasing the difficulty of operation and posing safety hazards.

[0003] The existing solutions are mainly:

[0004] 1) Adjust the camera's wide dynamic range: This method solves the problem of large differences in brightness and darkness by adjusting the camera's wide dynamic range. The camera's wide dynamic range is a method of global brightness adjustment. For scenes with strong contrast between light and dark, such as ship unloaders, the effect is usually not ideal. Not only can the dark areas not be improved, but it also brings problems such as increased noise and motion blur.

[0005] 2) Histogram equalization: Although this method can effectively improve the global contrast of the image, it is more suitable for processing images that are overall dark or bright and have low contrast. The effect is usually not obvious for scenes with large differences in light and dark in the grab operation of the ship unloader, and it will cause the loss of details in the dark (producing a mosaic-like phenomenon).

[0006] How to fundamentally solve the problem of strong contrast between light and dark between the deck and the cabin during the operation of the ship unloader, and improve the safety, efficiency and comfort of the grab ship unloader operation, is a technical problem that urgently needs to be solved. Summary of the invention

[0007] In order to at least solve the technical problems existing in the above-mentioned background technology, the present invention provides a visual improvement method, electronic device, storage medium and computer program product for the light contrast inside and outside the cabin of a large ship unloading machine.

[0008] The present invention provides a method for improving the visual contrast of light inside and outside the cabin of a large ship unloading machine, and the method comprises the following steps:

[0009] S1, receiving the real-time video stream collected by the surveillance camera;

[0010] S2. Performing inference calculation on the real-time video stream using an instance segmentation model to obtain hatch detection results and hatch size information;

[0011] S3, judging whether the proportion of the hatch in the video image exceeds a specified threshold according to the size information of the hatch, if so, go to S4, otherwise go to S1;

[0012] S4, calculating an adaptive gamma value according to the image brightness of the real-time video stream;

[0013] S5, performing gamma transformation on the image of the hatch area using the adaptive gamma value;

[0014] S6. Post-process the video image after the gamma transformation.

[0015] Optionally, step S2 further includes: performing normalization processing on the original image of the real-time video stream to eliminate noise and standardize image brightness.

[0016] Optionally, step S2 specifically includes:

[0017] Collecting image data containing various lighting conditions, ship types and hatch states, and annotating the image data respectively to obtain a training set, and using the training set to train the instance segmentation model;

[0018] The real-time video stream is input into the trained instance segmentation model, and the instance segmentation model performs inference calculation on the image frames in the real-time video stream to obtain a detection result of whether the image frames contain a hatch. If the hatch is contained, the position information and size information of the hatch are further determined.

[0019] Optionally, the instance segmentation model is a Mask R-CNN model.

[0020] Optionally, the specified threshold is 0.3.

[0021] Optionally, step S4 specifically includes:

[0022] Convert the image of the real-time video stream from the RGB color space to the HSV color space;

[0023] Calculate the average brightness of the image in the real-time video stream ;

[0024] Calculate the average brightness of the cabin area separately ;

[0025] Set a target brightness As target brightness;

[0026] The gamma value is calculated based on the deviation of the average brightness of the area from the target brightness:

[0027] .

[0028] Optionally, step S5 specifically includes:

[0029] Using the gamma change formula Calculate the output pixel value

[0030] In the formula, is the pixel value of the input image, is the pixel value of the output image, is the gamma value, is a constant.

[0031] Optionally, step S6 specifically includes:

[0032] The image of the real-time video stream is converted from the HSV color space back to the RGB space, and a color balance adjustment algorithm is used to eliminate possible color cast, and a noise suppression algorithm is used to remove noise in a low-light environment.

[0033] The present invention also provides an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method as described in any of the preceding items.

[0034] The present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, any of the above methods is executed.

[0035] The present invention also provides a computer program product, comprising a computer program stored on a non-transitory computer-readable medium, characterized in that: when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0036] The present invention introduces instance segmentation technology to accurately identify the hatch area, combines it with an adaptive gamma value calculation method, dynamically optimizes the contrast and brightness of the image in the cabin, and adds corresponding post-processing to solve the possible image distortion problem, thereby fundamentally solving the problem of strong light and dark contrast between the deck and the cabin during the operation of the ship unloader, and can improve the safety of the grab ship unloader operation, the efficiency of the operation and the comfort of the operator.

[0037] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0038] 1. Significantly enhanced visual clarity: Through adaptive adjustment of gamma values ​​and local gamma enhancement, the brightness and contrast of the cabin area are effectively improved. Even in low-light environments, the details of the cabin, such as cargo status and wall texture, can be clearly presented, providing operators with more intuitive visual information.

[0039] 2. Improve operational safety: The clear view inside the cabin reduces operational errors caused by poor visibility, such as preventing the grab from hitting the bulkhead or the equipment below, thereby reducing operational risks and improving the safety factor of the entire loading and unloading operation.

[0040] 3. Improve work efficiency: Reduce pauses for inspection or repeated operations due to insufficient light. Operators can quickly make judgments and operations directly through the enhanced image, greatly improving the continuity and efficiency of loading and unloading operations.

[0041] 4. Enhanced all-weather operation capability: In various lighting and weather conditions, this technology can ensure clear visual monitoring, so that ship unloading operations are no longer restricted by light conditions, realizing true all-weather operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 The present invention discloses a flow chart of a method for visually improving the light contrast inside and outside a large ship unloading cabin.

[0044] Figure 2 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0047] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, it is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the product of the invention is usually placed when used. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the system or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0048] The terms "first", "second", "third", and "fourth" in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order of the objects. For example, the first input, the second input, the third input, and the fourth input are used to distinguish different inputs rather than to describe a specific order of the inputs.

[0049] In the embodiments of the present invention, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0050] In the description of the embodiments of the present invention, unless otherwise specified, “multiple” means two or more than two. For example, multiple processing units refer to two or more processing units; multiple elements refer to two or more elements, etc.

[0051] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0052] like Figure 1 As shown, a method for visually improving the light contrast inside and outside the cabin of a large ship unloading engine room according to an embodiment of the present invention comprises the following steps:

[0053] S1, receiving the real-time video stream collected by the surveillance camera;

[0054] S2. Performing inference calculation on the real-time video stream using an instance segmentation model to obtain hatch detection results and hatch size information;

[0055] The instance segmentation model based on the advanced deep learning model is used to accurately segment the hatch area. This step can accurately distinguish the hatch from the surrounding environment, laying the foundation for subsequent local image enhancement, ensuring that only the dark part inside the cabin is processed, avoiding interference with the overly bright area outside the cabin.

[0056] S3, judging whether the proportion of the hatch in the video image exceeds a specified threshold according to the size information of the hatch, if so, go to S4, otherwise go to S1;

[0057] S4, calculating an adaptive gamma value according to the image brightness of the real-time video stream;

[0058] The present invention designs a dynamic gamma transformation function based on the brightness ratio between inside and outside the cabin, and achieves fine adaptation to different lighting conditions through logarithmic transformation of the brightness ratio. This adaptive mechanism can automatically adjust the gamma value according to the actual lighting conditions and optimize the image enhancement effect.

[0059] S5, performing gamma transformation on the image of the hatch area using the adaptive gamma value;

[0060] Using the local enhancement and global balance strategy, gamma transformation is applied only to the identified hatch area for local enhancement, keeping the image outside the cabin natural and avoiding the problem of over-enhancement or information distortion caused by global processing. At the same time, through subsequent color balance adjustment and noise suppression processing, the enhanced image is ensured to have natural colors and improve visual quality.

[0061] S6. Post-process the video image after the gamma transformation.

[0062] After the above processing, the operator can obtain clear images inside the cabin with appropriate contrast, which effectively reduces the visual pressure when working in complex lighting environments, improves judgment accuracy and work efficiency, and reduces potential errors caused by visual fatigue.

[0063] Optionally, step S2 further includes: performing normalization processing on the original image of the real-time video stream to eliminate noise and standardize image brightness.

[0064] To ensure that the image quality meets the processing requirements, the original images (each video image frame) in the real-time video stream obtained by the surveillance camera need to be denoised in preparation for subsequent automatic analysis. In addition, the image brightness needs to be standardized.

[0065] Optionally, step S2 specifically includes:

[0066] Collecting image data containing various lighting conditions, ship types and hatch states, and annotating the image data respectively to obtain a training set, and using the training set to train the instance segmentation model;

[0067] The real-time video stream is input into the trained instance segmentation model, and the instance segmentation model performs inference calculation on the image frames in the real-time video stream to obtain a detection result of whether the image frames contain a hatch. If the hatch is contained, the position information and size information of the hatch are further determined.

[0068] The instance segmentation model needs to be pre-trained offline using the training set before it can be put into use. The training set contains the collected image data of various lighting conditions, ship types, and hatch states, and the annotations are either manual or automatic to create the training set; then the selected instance segmentation model is used to train on the annotated data set, and the focus of the training process is on the accurate segmentation of the hatch area.

[0069] Optionally, the instance segmentation model is a Mask R-CNN model.

[0070] Optionally, the specified threshold is 0.3.

[0071] Optionally, step S4 specifically includes:

[0072] Converting the image of the real-time video stream from the RGB color space to the HSV color space, wherein the V value of a pixel represents the brightness of a single pixel;

[0073] Calculate the average brightness of the image in the real-time video stream , which is used to assess overall light levels;

[0074] Calculate the average brightness of the cabin area separately , which is used to assess the light level in the cabin;

[0075] Set a target brightness As the target brightness, this value can be adjusted according to the actual application scenario;

[0076] The gamma value is calculated based on the deviation of the average brightness of the area from the target brightness:

[0077] .

[0078] Optionally, step S5 specifically includes:

[0079] Using the gamma change formula Calculate the output pixel value

[0080] In the formula, is the pixel value of the input image (i.e., any image frame in a real-time video stream), is the pixel value of the output image, is the gamma value, is a constant (such as 255) used to ensure that the output value is within the appropriate range.

[0081] Optionally, step S6 specifically includes:

[0082] The image of the real-time video stream is converted from the HSV color space back to the RGB space, and a color balance adjustment algorithm is used to eliminate possible color cast, and a noise suppression algorithm is used to remove noise in a low-light environment.

[0083] This method dynamically optimizes the contrast and brightness of the cabin image, which can solve the problem of strong light and dark contrast between the deck and the cabin during the ship unloader operation to the greatest extent, but some subsequent processing steps are still required to further optimize the image quality and avoid distortion. Color balance adjustment algorithms (such as histogram equalization) are usually used to eliminate possible color casts and noise suppression algorithms (such as NLM) are used to remove noise in low-light environments.

[0084] Optionally, image data containing various lighting conditions, ship types, and hatch states are collected and annotated respectively to obtain a training set, including:

[0085] Collect image data containing various lighting conditions, ship types and hatch states, as well as collection time information corresponding to the image data;

[0086] Determine the annotation information associated with each image data, verify the annotation information using the corresponding acquisition time information, associate the verified annotation information with the corresponding image data, and delete the image data that fails the verification;

[0087] A training set is constructed based on all verified image data and corresponding annotation information.

[0088] Especially when the annotation information is automatically generated by a machine, the error probability of the annotation information will be very high. Therefore, the present invention collects the acquisition time information of these image data while collecting the image data. The acquisition time information can be used to verify the annotation information configured for each image data. Specifically, according to the acquisition time information of the image data and the time period after the cargo ship enters the operating area of ​​the grab ship unloader, the front part of the time period is the cargo ship entry period, during which the hatch is closed (to prevent seawater and rainwater from entering the cabin), and it is open in other time periods (mainly the unloading period). If the annotation information corresponding to the cargo ship entry period is "hatch exists", it is obviously wrong, and it is possible that the automatic annotation machine has a recognition error; and even if the automatic annotation machine also recognizes the closed hatch, the image data when the hatch is closed is not suitable for training the instance segmentation model. Therefore, the present invention deletes the image data in the cargo ship entry period and only uses the image data when the cargo ship hatch is open.

[0089] In addition, the image data during the cargo ship entry period and the corresponding annotation information "hatch exists" can be used as negative data, and the image data in other time periods can be used as positive data. The training data set is constructed by positive and negative data, which can improve the training effect.

[0090] The present invention also discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method as described in any of the preceding items.

[0091] The present invention also discloses a computer storage medium, on which a computer program is stored. When the computer program is run by a processor, any of the above methods is executed.

[0092] The present invention also discloses a computer program product, comprising a computer program stored on a non-transitory computer-readable medium, wherein the computer program implements any of the above methods when executed by a processor.

[0093] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0096] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for visual improvement of the light contrast inside and outside the cabin of a large ship unloading engine room, characterized in that: The method comprises the following steps: S1, receiving the real-time video stream collected by the surveillance camera; S2. Performing inference calculation on the real-time video stream using an instance segmentation model to obtain hatch detection results and hatch size information; S3, judging whether the proportion of the hatch in the video image exceeds a specified threshold according to the size information of the hatch, if so, go to S4, otherwise go to S1; S4, calculating an adaptive gamma value according to the image brightness of the real-time video stream; S5, performing gamma transformation on the image of the hatch area using the adaptive gamma value; S6, post-processing the video image after the gamma transformation; Step S4 specifically includes: Convert the image of the real-time video stream from the RGB color space to the HSV color space; Calculate the average brightness of the image in the real-time video stream ; Calculate the average brightness of the cabin area separately ; Set a target brightness As target brightness; The gamma value is calculated based on the deviation of the average brightness of the area from the target brightness: 。 2. A method for improving the visual contrast of light inside and outside the cabin of a large ship unloading engine room according to claim 1, characterized in that: Step S2 also includes: performing normalization processing on the original image of the real-time video stream to eliminate noise and standardize image brightness.

3. The method for improving the visual contrast of light inside and outside the cabin of a large ship unloading engine room according to claim 1, characterized in that: Step S2 specifically includes: Collecting image data containing various lighting conditions, ship types and hatch states, and annotating the image data respectively to obtain a training set, and using the training set to train the instance segmentation model; The real-time video stream is input into the trained instance segmentation model, and the instance segmentation model performs inference calculation on the image frames in the real-time video stream to obtain a detection result of whether the image frames contain a hatch. If the hatch is contained, the position information and size information of the hatch are further determined.

4. The method for improving the visual contrast of light inside and outside the cabin of a large ship unloading engine room according to claim 3, characterized in that: The instance segmentation model is a Mask R-CNN model.

5. The method for improving the visual contrast of light inside and outside the large ship unloading engine room according to claim 1, characterized in that: The specified threshold is 0.

3.

6. The method for improving the visual contrast of light inside and outside the large ship unloading cabin according to claim 1, characterized in that: Step S5 specifically includes: Using the gamma transform formula Calculate the output pixel value In the formula, is the pixel value of the input image, is the pixel value of the output image, is the gamma value, is a constant.

7. A method for improving the visual contrast of light inside and outside the cabin of a large ship unloading engine room according to claim 6, characterized in that: Step S6 specifically includes: The image of the real-time video stream is converted from the HSV color space back to the RGB space, and a color balance adjustment algorithm is used to eliminate possible color cast, and a noise suppression algorithm is used to remove noise in a low-light environment.

8. An electronic device comprising: A memory storing executable program code; A processor coupled to the memory; characterized in that: the processor calls the executable program code stored in the memory to execute the method according to any one of claims 1-7.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images