Image processing method and device, electronic equipment and computer storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a computer storage medium, which are applied to a monitoring terminal, and the monitoring terminal comprises a lens with a preset aperture value, a sensor with a preset pixel size and an intelligent image signal processor. The method comprises the following steps: collecting light through a lens with a preset aperture value; converting the light into an electric signal through a sensor with a preset pixel size; converting the electric signal into a color image frame through an intelligent image signal processor; and forming a monitoring picture based on the color image frame. By adopting the scheme, more light can be obtained and a high-definition color monitoring picture can be formed under low-light conditions such as night, monitoring details under the low-light conditions are effectively restored, and the definition of the monitoring picture is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image processing method, device, electronic device and computer storage medium. Background Art

[0002] With the continuous development of science and technology, the application scenarios and scope of monitoring systems are becoming more and more extensive. Whether the night scene can be truly restored is particularly important for the monitoring system.

[0003] In the related art, surveillance cameras usually use infrared night vision technology under low-light conditions, and the surveillance images taken are often in black and white, with blurred images, unable to restore the surveillance details in night scenes, and there is a problem of low clarity of the surveillance images. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device and computer storage medium, which can improve the clarity of monitoring images.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a monitoring terminal, wherein the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor; the method includes:

[0006] Collect light through a lens with a preset aperture value;

[0007] The light is converted into electrical signals by a sensor with a preset pixel size;

[0008] The electrical signal is converted into a color image frame by an intelligent image signal processor;

[0009] A monitoring screen is formed based on the color image frames.

[0010] In a possible implementation, converting the electrical signal into a color image frame by an intelligent image signal processor includes:

[0011] The electrical signal is converted into digital form by an intelligent image signal processor to obtain Bayer array data;

[0012] De-mosaicing is performed on the Bayer array data by using an intelligent image signal processor to obtain an initial color image frame;

[0013] The initial color image frame is subjected to color correction processing, noise reduction processing and image enhancement processing by an intelligent image signal processor to obtain a color image frame.

[0014] In a possible implementation, after the electrical signal is converted into a color image frame by the intelligent image signal processor, the method further includes:

[0015] In response to a first control instruction of a user, performing face recognition based on the color image frame;

[0016] When it is recognized that the color image frame includes a face target, a first reminder message is sent.

[0017] In a possible implementation, after the electrical signal is converted into a color image frame by the intelligent image signal processor, the method further includes:

[0018] In response to a second control instruction of the user, performing human figure detection based on the color image frame;

[0019] When a human-shaped target is detected in the color image frame, the direction and focal length of the lens with a preset aperture value are adjusted to track the human-shaped target.

[0020] In a possible implementation, the method further includes:

[0021] In response to a third control instruction from the user, cruise monitoring is performed within a preset time period at preset time intervals, and the various cruise angles of the cruise monitoring each correspond to a different light beam.

[0022] In a possible implementation, the monitoring terminal further includes at least two fill lights arranged in different directions;

[0023] The method also includes:

[0024] In response to a fourth control instruction from the user, the brightness of a target fill light among the at least two fill lights is adjusted to a target brightness to change the intensity of the light.

[0025] In a possible implementation, the monitoring terminal further includes a microphone and a speaker;

[0026] The method also includes:

[0027] In response to a fifth control instruction of the user, collecting a first sound source within a recording range of the monitoring terminal through a microphone, and sending a second reminder message based on the first sound source;

[0028] and / or

[0029] In response to a sixth control instruction of the user, a second sound source received by the monitoring terminal is played through the speaker; the second sound source is collected by a user terminal associated with the monitoring terminal.

[0030] In a second aspect, an embodiment of the present application provides a monitoring device, wherein the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor; the device includes:

[0031] A light collection module, used to collect light through a lens with a preset aperture value;

[0032] A first conversion module, used for converting light into an electrical signal through a sensor with a preset pixel size;

[0033] A second conversion module, used for converting the electrical signal into a color image frame through an intelligent image signal processor;

[0034] The picture forming module is used to form a monitoring picture based on the color image frame.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps provided in the first aspect of the embodiment of the present application.

[0036] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps provided in the first aspect of the embodiment of the present application.

[0037] The above-mentioned image processing method, device, electronic device and computer storage medium are applied to a monitoring terminal, which includes a lens with a preset aperture value, a sensor with a preset pixel size and an intelligent image signal processor. The monitoring terminal collects light through a lens with a preset aperture value, which can optimize the amount of light entering, help improve the brightness and contrast of the image, and facilitate the subsequent restoration of the color and details of the monitoring scene under low-light conditions based on the collected light; converting light into electrical signals through a sensor with a preset pixel size helps to improve the photosensitivity of each pixel unit, thereby improving the detail performance of the image under low-light conditions; converting electrical signals into color image frames through an intelligent image signal processor, and forming a monitoring screen based on the color image frames, helps to remove noise in the image, ensure that the image maintains natural colors under low-light conditions, and improve the image quality and the presentation effect of the monitoring screen. The use of this solution can obtain more light under low-light conditions such as at night and form a high-definition color monitoring screen, effectively restore the monitoring details under low-light conditions, and improve the clarity of the monitoring screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of an image processing system provided as an exemplary embodiment of the present application;

[0040] Figure 2 A flowchart of an image processing method provided by an exemplary embodiment of the present application;

[0041] Figure 3 A flowchart of another image processing method provided by an exemplary embodiment of the present application;

[0042] Figure 4 A flowchart of another image processing method provided by an exemplary embodiment of the present application;

[0043] Figure 5 A schematic diagram of a monitoring terminal including a fill light provided for an exemplary embodiment of the present application;

[0044] Figure 6 A schematic diagram of a fill light brightness adjustment page provided by an exemplary embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a monitoring device provided by an exemplary embodiment of the present application;

[0046] Figure 8 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0049] First, an application scenario applicable to the embodiment of the present application is introduced: a certain intelligent surveillance camera can be installed at the door of the house to monitor the entry and exit of people or the movement of people to improve family safety. The intelligent surveillance camera is equipped with a lens with a preset aperture value (such as an F1.0 lens), a sensor with a preset pixel size (such as a sensor with a pixel size of 2.0μm), and an intelligent image signal processor. Among them, the lens with a preset aperture value is used to collect light and focus the collected light onto a sensor with a preset pixel size; the sensor with a preset pixel size is used to convert the received light into an electrical signal; the intelligent image signal processor is used to convert the electrical signal into a color image frame, and form a monitoring screen based on the color image frame.

[0050] In addition, the smart surveillance camera is also equipped with a camera monitoring APP (Application) installed on the mobile phone. Users can use the APP to view the monitoring images uploaded by the smart surveillance camera in real time, and realize two-way voice calls and other functions with the smart surveillance camera through the APP.

[0051] The image processing method provided in the embodiment of the present application can be applied to Figure 1 In the monitoring system shown. The monitoring system includes a monitoring terminal 11, a user terminal 12 and a server 13. The monitoring terminal 11 can be connected to the user terminal 12 wirelessly (such as Bluetooth, Wi-Fi, etc.) or wiredly, and the monitoring terminal 11 and the user terminal 12 can communicate with the server 13 through the network. The data storage system can be integrated on the server 13, or it can be placed on the cloud or other network servers. Among them, the monitoring terminal 11 includes a lens with a preset aperture value, a sensor with a preset pixel size, an intelligent image signal processor, at least two fill lights set in different directions, a microphone and a speaker. The monitoring terminal 11 can be, but not limited to, various intelligent monitoring devices, etc. The user terminal 12 is installed with an APP that matches the monitoring terminal 11, and the user terminal 12 can be, but not limited to, various smart phones, tablet computers, laptops, personal computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 13 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0052] Optionally, the monitoring terminal 11 collects light through a lens with a preset aperture value; converts the light into an electrical signal through a sensor with a preset pixel size; converts the electrical signal into a color image frame through an intelligent image signal processor; and forms a monitoring screen based on the color image frame. The monitoring terminal 11 then sends the monitoring screen to the server 13. The user terminal 12 loads the monitoring screen uploaded by the monitoring terminal 11 from the server 13 through the network and displays it on the user interface.

[0053] In one embodiment, Figure 2 As shown, an image processing method is provided, which is applied to Figure 1 The monitoring terminal 11 shown is used as an example for description, and the following steps are included:

[0054] S201: Collecting light through a lens with a preset aperture value.

[0055] Among them, the lens with a preset aperture value can be a lens with a larger aperture value, such as an F1.0 lens, so as to ensure that the monitoring terminal 11 can collect sufficient light even under low light conditions.

[0056] It can be understood that the aperture value (i.e., F value) is a way of expressing the aperture of the lens, indicating the maximum aperture size of the lens. Since the aperture value is the reciprocal of the aperture of the lens, the smaller the F value, the larger the aperture and the more light entering.

[0057] Optionally, the monitoring terminal 11 collects ambient light through an F1.0 lens, which not only helps it to obtain as much light as possible within the monitoring range and enhance the brightness of the image, but also helps to improve the exposure effect of the image to enhance the contrast of the image. Whether in a well-lit environment such as daytime or a low-light environment such as night, the monitoring terminal 11 can effectively restore the color and details of the monitoring scene based on the light collected by the lens, thereby adapting to different lighting conditions to provide an image closer to the real scene.

[0058] In this embodiment, the monitoring terminal collects light through a lens with a preset aperture value, which can optimize the amount of light entering, so that it can enhance the brightness and contrast of the image according to the light collected by the lens, facilitate the subsequent restoration of the color and details of the monitoring scene, and improve the clarity of the monitoring image under different lighting conditions, especially low light conditions.

[0059] S202: Convert the light into an electrical signal through a sensor with a preset pixel size.

[0060] The sensor with the preset pixel size can use a sensor with a larger pixel size, such as a sensor with a pixel size of 2.0 μm, so as to improve the detail performance of the image under low light conditions. The electrical signal specifically refers to an analog electrical signal.

[0061] It can be understood that pixel size refers to the physical size of each pixel unit. A larger pixel size means that each pixel unit can capture more light, that is, each pixel unit can receive more photons and generate a stronger electrical signal, thereby effectively reducing image noise and making image details clearer.

[0062] Optionally, the monitoring terminal 11 converts the light focused by the F1.0 lens into an analog electrical signal through a sensor with a pixel size of 2.0 μm, so that the pixel unit of the sensor can capture more photons under different lighting conditions, especially low light conditions, thereby reducing image noise and improving the image details.

[0063] In this embodiment, the monitoring terminal converts the light collected by a lens with a preset aperture value into an analog electrical signal through a sensor with a preset pixel size, which helps to improve the photosensitivity of each pixel unit, thereby improving the details of the image, and further improving the clarity of the monitoring image under different lighting conditions, especially low light conditions.

[0064] S203: Convert the electrical signal into a color image frame through an intelligent image signal processor.

[0065] Among them, the Artificial Intelligence Image Signal Processor (AI-ISP) combines artificial intelligence technology with traditional image signal processing technology, and can generate high-quality images based on the raw data obtained from the sensor with a preset pixel size. Specifically, the intelligent image signal processor can use other artificial intelligence technologies such as deep learning to improve the quality of the image, such as color correction, noise reduction, image enhancement, etc.

[0066] Optionally, the monitoring terminal 11 performs analog-to-digital conversion on the analog electrical signal through an intelligent image signal processor to obtain digital signal data arranged in a Bayer array (i.e., Bayer array data), and then performs de-mosaicing on the Bayer array data through the intelligent image signal processor to obtain an initial color image frame. Finally, the initial color image frame is subjected to color correction, noise reduction, and image enhancement processing through the intelligent image signal processor to obtain a color image frame.

[0067] In this embodiment, the monitoring terminal obtains analog electrical signals from a sensor of a preset pixel size through an intelligent image signal processor, and converts the analog electrical signals into color image frames through digital-to-analog conversion, de-mosaicing, color correction, noise reduction and image enhancement processing, which helps to remove noise in the image, ensure that the image maintains natural color under low light conditions, and effectively restore the color and details of the image, thereby improving the clarity of the monitoring screen under different lighting conditions, especially low light conditions.

[0068] S204: forming a monitoring screen based on the color image frame.

[0069] Optionally, the monitoring terminal 11 encodes the color image frame formed based on the above steps into a standard video format (such as H.264, H.265, etc.), and sends the encoded color image frame to the server. The user terminal can obtain the color image frame in the standard video format from the server through the network, and display the color image frame in the standard video format as a monitoring screen on the user interface.

[0070] It is worth noting that this embodiment only describes a preferred implementation of the present application. After the monitoring terminal 11 encodes the color image frame formed based on the above steps into a standard video format, it can also send the encoded color image frame to the user terminal wirelessly (such as Bluetooth, Wi-Fi, etc.) or wired, so that the user terminal can display the encoded color image frame as a monitoring screen on the user interface.

[0071] In this embodiment, the monitoring terminal forms a monitoring screen based on the color image frame formed in the above steps, and transmits the monitoring screen to the user terminal for display, so that the user can perform real-time monitoring through the high-definition color monitoring screen, thereby improving the monitoring effect and the user experience.

[0072] The above-mentioned image processing method is applied to a monitoring terminal including a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor. Specifically, the monitoring terminal collects light through a lens with a preset aperture value, which can optimize the amount of light entering, and help improve the brightness and contrast of the image; the sensor with a preset pixel size converts the light collected through the lens with a preset aperture value into an analog electrical signal, which helps to improve the photosensitivity of each pixel unit, thereby improving the detail performance of the image; the intelligent image signal processor obtains the analog electrical signal from the sensor with a preset pixel size, and converts the analog electrical signal into a color image frame through digital-to-analog conversion processing, de-mosaic processing, color correction processing, noise reduction processing, and image enhancement processing, which helps to remove noise in the image and ensure that the image maintains natural color under low light conditions, thereby effectively restoring the color and details of the image; a monitoring screen is formed based on the color image frame, and the monitoring screen is transmitted to the user terminal for display, so that the user can monitor in real time through a high-definition color monitoring screen, thereby improving the monitoring effect and the user's experience. This solution can obtain more light under different lighting conditions, especially low-light conditions at night, and form a high-definition color monitoring picture, effectively restoring the monitoring details under low-light conditions, improving the clarity of the monitoring picture, and thus improving the monitoring effect and user experience.

[0073] In one embodiment, Figure 3 As shown, another image processing method is provided, which is applied to Figure 1The monitoring terminal 11 shown is used as an example for description, and the following steps are included:

[0074] S301: Collecting light through a lens with a preset aperture value.

[0075] Specifically, S301 is consistent with S201 and will not be described in detail here.

[0076] S302: Convert the light into an electrical signal through a sensor with a preset pixel size.

[0077] Specifically, S302 is consistent with S202 and will not be described in detail here.

[0078] S303: Perform analog-to-digital conversion on the electrical signal through an intelligent image signal processor to obtain Bayer array data.

[0079] The Bayer array data refers to digital signal data arranged in a Bayer array.

[0080] Optionally, the monitoring terminal 11 performs analog-to-digital conversion processing on the analog electrical signal obtained from the sensor of a preset pixel size through an intelligent image signal processor to obtain digital signal data arranged in a Bayer array.

[0081] S304: De-mosaicing the Bayer array data by using an intelligent image signal processor to obtain an initial color image frame.

[0082] Among them, the initial color image frame is a complete RGB image.

[0083] It can be understood that each pixel in the Bayer array contains only one color information (red, green or blue). In order to convert the Bayer array data into a complete RGB image, the intelligent image signal processor needs to perform demosaicing on the Bayer array, that is, to estimate the other two color information of each pixel through an interpolation algorithm. For example, for a red pixel, the intelligent image signal processor needs to estimate its corresponding green and blue values; for a green pixel, the intelligent image signal processor needs to estimate its corresponding red and blue values.

[0084] S305: Performing color correction processing, noise reduction processing and image enhancement processing on the initial color image frame through an intelligent image signal processor to obtain a color image frame.

[0085] The color correction processing includes but is not limited to white balance correction processing, color correction processing, gamma correction processing, etc.

[0086] Optionally, the monitoring terminal 11 performs intelligent white balance correction, color correction, gamma correction, noise reduction and image enhancement processing on the initial color image frame through an intelligent image signal processor, so that the initial color image frame is clearer and the color is more natural, and the image frame optimized by the intelligent image signal processor is used as the final color image frame for forming the monitoring screen.

[0087] In this embodiment, the monitoring terminal obtains analog electrical signals from a sensor of a preset pixel size through an intelligent image signal processor, and converts the analog electrical signals into color image frames through digital-to-analog conversion, de-mosaicing, color correction, noise reduction and image enhancement processing, which helps to remove noise in the image, ensure that the image maintains natural color under low light conditions, and effectively restore the color and details of the image, thereby improving the clarity of the monitoring screen under different lighting conditions, especially low light conditions.

[0088] S306: In response to a first control instruction from the user, performing face recognition based on the color image frame.

[0089] The first control instruction is used to enable the face recognition function of the monitoring terminal 11. The control form of the first control instruction includes but is not limited to mobile APP control form, voice control form, gesture control form, key control form, etc.

[0090] Optionally, the user controls the monitoring terminal 11 through a user terminal, and the user terminal is installed with an APP that matches the monitoring terminal 11. Exemplarily, the user turns on the face recognition function of the monitoring terminal 11 based on the function setting page of the APP on the user terminal (i.e., issues a first control instruction through the APP on the user terminal). The monitoring terminal 11 responds to the first control instruction issued by the user through the user terminal and performs face recognition based on the color image frame formed by the above steps.

[0091] S307: When a face target is identified in the color image frame, a first reminder message is sent.

[0092] The face target is a face region including face features. The first reminder message is a reminder message generated by the monitoring terminal 11 based on the identified face target.

[0093] Optionally, when the monitoring terminal 11 recognizes that the color image frame includes a face area, it generates a first reminder message based on the recognized face target, such as "someone appears at the door", and sends the first reminder message to the server. The user terminal can load the first reminder message from the server through the network and display it on the user interface.

[0094] It is worth noting that this embodiment only describes a preferred implementation of the present application. After the monitoring terminal 11 generates a first reminder message based on the identified face target, it can also send the first reminder message directly to the user terminal via wireless means (such as Bluetooth, Wi-Fi, etc.) or wired means, so that the user terminal can display the monitoring screen on the user interface.

[0095] In this embodiment, after the user turns on the face recognition function of the monitoring terminal, the monitoring terminal performs face recognition based on the color image frame, and sends a first reminder message when a face target is recognized in the color image frame, which helps the user to timely understand the entry and exit of people in the monitoring area based on the first reminder message. The above process of performing face recognition based on color image frames realizes intelligent monitoring and improves the monitoring effect and user experience.

[0096] S308: In response to a second control instruction of the user, perform human figure detection based on the color image frame.

[0097] The second control instruction is used to enable the human detection function of the monitoring terminal 11. The control form of the second control instruction includes but is not limited to mobile APP control form, voice control form, gesture control form, key control form, etc.

[0098] Optionally, the user controls the monitoring terminal 11 through a user terminal, and the user terminal is installed with an APP that matches the monitoring terminal 11. Exemplarily, the user turns on the human detection function of the monitoring terminal 11 based on the function setting page of the APP on the user terminal (i.e., issues a second control instruction through the APP on the user terminal). The monitoring terminal 11 responds to the second control instruction issued by the user through the user terminal and performs human detection based on the color image frame formed by the above steps.

[0099] S309: When it is detected that the color image frame includes a human-shaped target, adjust the direction and focal length of the lens of the preset aperture value to track the human-shaped target.

[0100] The humanoid target is a humanoid region containing humanoid features.

[0101] Optionally, when the monitoring terminal 11 recognizes that the color image frame includes a human-shaped area, it adjusts the direction of a lens with a preset aperture value (such as an F1.0 lens) based on the position of the detected human-shaped target in the color image frame, so that the human-shaped target can be captured by the lens with a preset aperture value; and based on the size of the area occupied by the detected human-shaped target in the color image frame, it determines the distance of the human-shaped target from the monitoring terminal 11, and adjusts the focal length of the lens with a preset aperture value according to the distance, thereby tracking the human-shaped target.

[0102] In this embodiment, after the user turns on the human shape detection function of the monitoring terminal, the monitoring terminal performs human shape detection based on the color image frame, and when a human shape target is detected in the color image frame, the direction and focal length of the lens with a preset aperture value are adjusted to track the human shape target, which helps the monitoring user to timely understand the flow of people in the monitoring area. The above process of performing human shape detection based on color image frames realizes intelligent monitoring and improves the monitoring effect and user experience.

[0103] S310: In response to a third control instruction of the user, cruise monitoring is performed within a preset time period at a preset time interval; the various cruise angles of the cruise monitoring respectively correspond to different light beams.

[0104] Among them, the third control instruction is used to turn on the timed cruise function of the monitoring terminal 11. The control form of the third control instruction includes but is not limited to mobile APP control form, voice control form, gesture control form, button control form, etc. The preset time period can be a time period preset by the user, such as 0:00 to 6:00 in the morning from Monday to Friday. The preset time interval can be a time interval preset by the user, such as 30 minutes, 1 hour, etc. The shorter the preset time interval is set, the more frequent the cruise of the monitoring terminal 11.

[0105] Optionally, the user controls the monitoring terminal 11 through a user terminal, and the user terminal is installed with an APP that matches the monitoring terminal 11. Exemplarily, based on the function setting page of the APP on the user terminal, the user turns on the timed cruise function of the monitoring terminal 11 (i.e., issues a third control instruction through the APP on the user terminal), and completes the setting of the cruise time period and the cruise time interval on the page. The monitoring terminal 11 responds to the third control instruction issued by the user through the user terminal, and performs cruise monitoring within a preset time period and at a preset time interval. Specifically, when performing cruise monitoring, the monitoring terminal 11 can change its cruise angle by adjusting the direction of a lens with a preset aperture value (such as an F1.0 lens), thereby comprehensively covering every corner of the monitoring area.

[0106] It is understandable that the monitoring terminal 11 can not only adjust the horizontal direction of the lens of the preset aperture value to achieve horizontal cruising to cover the horizontal monitoring area, but also adjust the vertical direction of the lens of the preset aperture value to achieve vertical cruising to cover the vertical monitoring area. In addition, the monitoring terminal 11 can also adjust the horizontal and vertical directions of the lens of the preset aperture value at the same time to achieve multi-angle cruising to cover the monitoring areas in different directions.

[0107] In this embodiment, after the user turns on the timed cruise function of the monitoring terminal, the monitoring terminal performs cruise monitoring at preset time intervals within a preset time period, which not only comprehensively covers every corner of the monitoring area, but also avoids the situation where visual blind spots cannot be effectively monitored due to insufficient or uneven light distribution, thereby realizing intelligent monitoring and improving the monitoring effect and user experience.

[0108] The above-mentioned image processing method obtains analog electrical signals from a sensor of a preset pixel size through an intelligent image signal processor, and converts the analog electrical signals into color image frames through digital-to-analog conversion processing, de-mosaic processing, color correction processing, noise reduction processing and image enhancement processing, which helps to remove noise in the image, ensure that the image maintains natural color under low light conditions, and effectively restore the color and details of the image, thereby improving the clarity of the monitoring picture under different lighting conditions, especially under low light conditions; face recognition and human shape detection are performed based on color image frames, realizing intelligent monitoring; in response to the user's third control instruction, cruise monitoring is performed at preset time intervals within a preset time period, which can not only comprehensively cover every corner of the monitoring area, but also avoid the situation that the visual blind area cannot be effectively monitored due to insufficient or uneven light distribution based on the different light corresponding to different cruise angles, thereby improving the monitoring effect. The adoption of this scheme can obtain as much light as possible under low light conditions such as night and form high-definition color image frames, effectively restoring the monitoring details under low light conditions, and in addition, in response to the user's control instruction, intelligent monitoring is realized based on the acquired color image frames, thereby improving the monitoring effect and user experience.

[0109] In one embodiment, Figure 4 As shown, another image processing method is provided, which is applied to Figure 1 The monitoring terminal 11 shown is used as an example for description, and the following steps are included:

[0110] S401: Collecting light through a lens with a preset aperture value.

[0111] Specifically, S401 is consistent with S201 and will not be described in detail here.

[0112] S402: Convert the light into an electrical signal through a sensor with a preset pixel size.

[0113] Specifically, S402 is consistent with S202 and will not be repeated here.

[0114] S403: Convert the electrical signal into a color image frame through an intelligent image signal processor.

[0115] Specifically, S403 is consistent with S203 and will not be described in detail here.

[0116] S404: forming a monitoring screen based on the color image frame.

[0117] Specifically, S404 is consistent with S204 and will not be described in detail here.

[0118] S405: In response to a fourth control instruction of the user, the brightness of a target fill light among the at least two fill lights is adjusted to a target brightness to change the intensity of the light.

[0119] Optionally, the monitoring terminal 11 includes at least two fill lights arranged in different directions, and the at least two fill lights can provide additional light sources under low light conditions, improve the brightness, contrast and detail expression of the image, and thus improve the clarity of the monitoring picture. The fourth control instruction is used to adjust the brightness of the target fill light selected by the user among the at least two fill lights to the target brightness set by the user, thereby changing the light intensity collected by the lens of the preset aperture value. It can be understood that the target fill light selected by the user can be one or more, and the target brightness is the brightness percentage corresponding to the target fill light. The control form of the fourth control instruction includes but is not limited to mobile APP control form, voice control form, gesture control form, button control form, etc.

[0120] For example, see Figure 5 The monitoring terminal 11 includes a front fill light 51 and a rear fill light 52 arranged in the front and rear directions. The front fill light 51 is mainly used to illuminate the monitoring area in front of the monitoring terminal 11, while the rear fill light 52 is used to illuminate the monitoring area on the back of the monitoring terminal 11 for auxiliary lighting, ensuring that the monitoring terminal 11 can obtain sufficient light at different angles and in different environments. The monitoring terminal 11 can automatically control the opening or closing of the fill light according to the light collected by the lens of the preset aperture value, and the user can also control the monitoring terminal 11 through a user terminal installed with an APP matching the monitoring terminal 11.

[0121] See also Figure 6 , the user can adjust the brightness of the fill light of the monitoring terminal 11 based on the real-time monitoring screen displayed in the user interface of the user terminal. For example, the user can select the target fill light to be controlled by clicking the selection control 61 (the front fill light in the figure), control the on or off of the target fill light by clicking the switch control 62, and adjust the brightness of the target fill light to the target brightness (the target brightness percentage in the figure is 67%) by sliding the brightness adjustment slider 63, thereby changing the light intensity collected by the lens of the preset aperture value to ensure that the user can clearly see the details in the monitoring screen.

[0122] In this embodiment, the monitoring terminal adjusts the brightness of the target fill light among at least two fill lights to the target brightness in response to the user's fourth control instruction to change the intensity of the light. This enables the user to adjust the brightness of each fill light according to actual needs to adapt to different monitoring scenarios, thereby improving the monitoring effect and flexibility.

[0123] S406: In response to the fifth control instruction of the user, a first sound source within the recording range of the monitoring terminal is collected through a microphone, and a second reminder message is sent based on the first sound source.

[0124] Among them, the fifth control instruction is used to enable the voice collection function of the monitoring terminal 11. The control form of the fifth control instruction includes but is not limited to voice control form, gesture control form, button control form, mobile APP control form, etc. The second reminder message is a reminder message generated by the monitoring terminal 11 based on the collected first sound source, and the reminder form of the second reminder message includes but is not limited to voice reminder form, text reminder form, etc.

[0125] Optionally, the user may issue a fifth control instruction to the monitoring terminal 11 in the form of voice control. Exemplarily, the user may automatically wake up the voice collection function of the monitoring terminal 11 (i.e., issue the fifth control instruction in the form of voice control) by inputting voice within the recording range of the monitoring terminal 11. In response to the fifth control instruction issued by the user through the user terminal, the monitoring terminal 11 collects the first sound source within the recording range through the built-in microphone, and sends the second reminder message based on the first sound source.

[0126] Understandably, when the courier delivers the item to the doorstep but fails to contact the recipient, the courier can input voice near the monitoring terminal 11 to automatically wake up the voice collection function of the monitoring terminal 11 and leave a message. The monitoring terminal 11 generates a second reminder message in voice or text form based on the courier's message, and sends it to the user terminal (such as a mobile phone) installed with the APP supporting the monitoring terminal 11, so that the recipient can take the courier at the doorstep as soon as possible after receiving the second reminder message.

[0127] S407: In response to a sixth control instruction of the user, a second sound source received by the monitoring terminal is played through a speaker; the second sound source is collected by a user terminal associated with the monitoring terminal.

[0128] The sixth control instruction is used to enable the voice broadcast function of the monitoring terminal 11. The control form of the sixth control instruction includes but is not limited to mobile APP control form, voice control form, gesture control form, key control form, etc.

[0129] Optionally, the user can control the monitoring terminal 11 through a user terminal, which is installed with an APP that matches the monitoring terminal 11. Exemplarily, based on the function setting page of the APP on the user terminal, the user turns on the voice broadcast function of the monitoring terminal 11 (i.e., issues the sixth control instruction through the APP on the user terminal), and collects the second sound source through the user terminal. After the user terminal sends the collected second sound source to the monitoring terminal 11, the monitoring terminal 11 responds to the sixth control instruction issued by the user through the user terminal and plays the second sound source received by the monitoring terminal through the speaker.

[0130] It is understandable that when the courier delivers the item to the doorstep but fails to contact the recipient, he can input voice near the monitoring terminal 11 to automatically wake up the voice collection function of the monitoring terminal 11 and leave a message. The monitoring terminal 11 generates a second reminder message in voice or text form based on the courier's message, and sends it to a user terminal (such as a mobile phone) that has an APP supporting the monitoring terminal 11 installed. After the recipient sees the second reminder message, he can turn on the voice broadcast function of the monitoring terminal 11 based on the function setting page of the APP on the user terminal, and record the message through the user terminal. After the user terminal sends the recording to the monitoring terminal 11, the monitoring terminal 11 plays the recording through the speaker to remind the courier that the recipient knows that the courier has been delivered.

[0131] In this embodiment, the monitoring terminal responds to the user's fifth control instruction, collects the first sound source within the recording range of the monitoring terminal through the microphone, and sends a second reminder message based on the first sound source; responds to the user's sixth control instruction, plays the second sound source received by the monitoring terminal through the speaker, enabling the user to adjust the sound collection and playback settings according to actual needs, thereby realizing a two-way voice call function and improving the monitoring effect and flexibility.

[0132] The above-mentioned image processing method, by responding to the fourth control instruction of the user, adjusts the brightness of the target fill light among at least two fill lights to the target brightness to change the intensity of the light, so that the user can adjust the brightness of each fill light according to actual needs to adapt to different monitoring scenes. In response to the fifth control instruction of the user, the first sound source within the recording range of the monitoring terminal is collected by the microphone, and a second reminder message is sent based on the first sound source; in response to the sixth control instruction of the user, the second sound source received by the monitoring terminal is played through the speaker, so that the user can adjust the sound collection and playback settings according to actual needs, thereby realizing a two-way voice call function. The adoption of this solution can not only obtain as much light as possible under low-light conditions such as at night and form a high-definition color image frame to improve the clarity of monitoring, but also effectively improve the monitoring effect and monitoring flexibility by independently controlling the fill lights, collecting voices on the monitoring terminals, and controlling voice broadcasting.

[0133] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0134] Based on the inventive concept of the above-mentioned image processing method, Figure 7 As shown, the embodiment of the present application further provides an image processing device 700 for implementing the above-mentioned image processing method. The image processing device 700 is applied to a monitoring terminal, and the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor. The image processing device 700 includes:

[0135] A light collection module 701, used to collect light through a lens with a preset aperture value;

[0136] A first conversion module 702, configured to convert light into an electrical signal through a sensor of a preset pixel size;

[0137] The second conversion module 703 is used to convert the electrical signal into a color image frame through an intelligent image signal processor;

[0138] The picture forming module 704 is used to form a monitoring picture based on the color image frame.

[0139] In a possible implementation, the second conversion module 703 is specifically used to perform analog-to-digital conversion processing on the electrical signal through an intelligent image signal processor to obtain Bayer array data; perform demosaicing processing on the Bayer array data through the intelligent image signal processor to obtain an initial color image frame; and perform color correction processing, noise reduction processing, and image enhancement processing on the initial color image frame through the intelligent image signal processor to obtain a color image frame.

[0140] In a possible implementation, the image processing device 700 further includes a response module for performing face recognition based on the color image frame in response to a first control instruction of the user; and sending a first reminder message when a face target is recognized in the color image frame.

[0141] In one possible implementation, the response module is also used to respond to a second control instruction from the user and perform human shape detection based on the color image frame; when a human shape target is detected in the color image frame, the direction and focal length of the lens with a preset aperture value are adjusted to track the human shape target.

[0142] In a possible implementation, the response module is further configured to respond to a third control instruction of the user to perform cruise monitoring at preset time intervals within a preset time period; and the various cruise angles of the cruise monitoring each correspond to a different light beam.

[0143] In a possible implementation, the monitoring terminal also includes at least two fill lights arranged in different directions; the above-mentioned response module is also used to adjust the brightness of the target fill light among the at least two fill lights to the target brightness in response to the user's fourth control instruction to change the intensity of the light.

[0144] In one possible implementation, the monitoring terminal also includes a microphone and a speaker; the above-mentioned response module is also used to respond to the user's fifth control instruction, collect the first sound source within the recording range of the monitoring terminal through the microphone, and send a second reminder message based on the first sound source; and / or respond to the user's sixth control instruction, play the second sound source received by the monitoring terminal through the speaker; the second sound source is collected by the user terminal associated with the monitoring terminal.

[0145] Each module in the above-mentioned image processing device 700 can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0146] The embodiment of the present application also provides an electronic device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external user terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0147] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In a possible implementation, an electronic device is provided, which is applied to a monitoring terminal. The monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor. The electronic device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0149] Collect light through a lens with a preset aperture value;

[0150] The light is converted into electrical signals by a sensor with a preset pixel size;

[0151] The electrical signal is converted into a color image frame by an intelligent image signal processor;

[0152] A monitoring screen is formed based on the color image frames.

[0153] In a possible implementation, when the processor executes the computer program, the following steps are further implemented: performing analog-to-digital conversion processing on the electrical signal through an intelligent image signal processor to obtain Bayer array data; performing demosaicing processing on the Bayer array data through the intelligent image signal processor to obtain an initial color image frame; performing color correction processing, noise reduction processing and image enhancement processing on the initial color image frame through the intelligent image signal processor to obtain a color image frame.

[0154] In a possible implementation, the processor further implements the following steps when executing the computer program: in response to a first control instruction from the user, performing face recognition based on the color image frame; and sending a first reminder message when a face target is recognized to be included in the color image frame.

[0155] In one possible implementation, the processor further implements the following steps when executing the computer program: in response to a second control instruction from the user, performing human figure detection based on the color image frame; and when a human figure target is detected in the color image frame, adjusting the direction and focal length of the lens with a preset aperture value to track the human figure target.

[0156] In a possible implementation, the processor further implements the following steps when executing the computer program: in response to a third control instruction of the user, cruise monitoring is performed at preset time intervals within a preset time period; and the light corresponding to each of the multiple cruise angles of the cruise monitoring is different.

[0157] In one possible implementation, the monitoring terminal also includes at least two fill lights arranged in different directions, and the processor also implements the following steps when executing the computer program: in response to the user's fourth control instruction, the brightness of the target fill light among the at least two fill lights is adjusted to the target brightness to change the intensity of the light.

[0158] In one possible implementation, the monitoring terminal also includes a microphone and a speaker, and the processor also implements the following steps when executing the computer program: in response to the user's fifth control instruction, the first sound source within the recording range of the monitoring terminal is collected through the microphone, and a second reminder message is sent based on the first sound source; and / or in response to the user's sixth control instruction, the second sound source received by the monitoring terminal is played through the speaker; the second sound source is collected by a user terminal associated with the monitoring terminal.

[0159] The present application also provides a computer storage medium, which stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes one or more steps in the above embodiment. If the components of the above electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer readable storage medium.

[0160] In a possible implementation, a computer storage medium is provided, which is applied to a monitoring terminal, wherein the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor. A computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Collect light through a lens with a preset aperture value;

[0162] The light is converted into electrical signals by a sensor with a preset pixel size;

[0163] The electrical signal is converted into a color image frame by an intelligent image signal processor;

[0164] A monitoring screen is formed based on the color image frames.

[0165] In one possible implementation, when the computer program is executed by the processor, the following steps are also implemented: performing analog-to-digital conversion processing on the electrical signal through an intelligent image signal processor to obtain Bayer array data; performing demosaicing processing on the Bayer array data through the intelligent image signal processor to obtain an initial color image frame; performing color correction processing, noise reduction processing and image enhancement processing on the initial color image frame through the intelligent image signal processor to obtain a color image frame.

[0166] In one possible implementation, when the computer program is executed by the processor, the following steps are also implemented: in response to a first control instruction from the user, face recognition is performed based on the color image frame; and when a face target is identified in the color image frame, a first reminder message is sent.

[0167] In one possible implementation, when the computer program is executed by the processor, the following steps are also implemented: in response to a second control instruction from the user, human figure detection is performed based on the color image frame; when a human figure target is detected in the color image frame, the direction and focal length of the lens with a preset aperture value are adjusted to track the human figure target.

[0168] In a possible implementation, when the computer program is executed by the processor, the following steps are also implemented: in response to a third control instruction of the user, cruise monitoring is performed at preset time intervals within a preset time period; and the light corresponding to each of the multiple cruise angles of the cruise monitoring is different.

[0169] In one possible implementation, the monitoring terminal also includes at least two fill lights arranged in different directions, and the computer program, when executed by the processor, also implements the following steps: in response to a fourth control instruction of the user, adjusting the brightness of a target fill light among the at least two fill lights to a target brightness to change the intensity of the light.

[0170] In one possible implementation, the monitoring terminal also includes a microphone and a speaker, and the computer program, when executed by the processor, also implements the following steps: in response to the user's fifth control instruction, the first sound source within the recording range of the monitoring terminal is collected through the microphone, and a second reminder message is sent based on the first sound source; and / or in response to the user's sixth control instruction, the second sound source received by the monitoring terminal is played through the speaker; the second sound source is collected by a user terminal associated with the monitoring terminal.

[0171] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer storage medium or transmitted through the computer storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)). It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0173] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims.

[0174] The above describes a specific embodiment of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An image processing method, characterized in that: Applied to a monitoring terminal, the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor; the method includes: Collecting light through the lens with the preset aperture value; Converting the light into an electrical signal through the sensor of the preset pixel size; converting the electrical signal into a color image frame by the intelligent image signal processor; A monitoring screen is formed based on the color image frame.

2. The method according to claim 1, characterized in that The step of converting the electrical signal into a color image frame by the intelligent image signal processor comprises: Performing analog-to-digital conversion on the electrical signal by the intelligent image signal processor to obtain Bayer array data; Performing demosaicing on the Bayer array data by the intelligent image signal processor to obtain an initial color image frame; The intelligent image signal processor performs color correction processing, noise reduction processing and image enhancement processing on the initial color image frame to obtain the color image frame.

3. The method according to claim 1, characterized in that After the intelligent image signal processor converts the electrical signal into a color image frame, the method further includes: In response to a first control instruction of a user, performing face recognition based on the color image frame; When it is recognized that the color image frame includes a face target, a first reminder message is sent.

4. The method according to claim 1, characterized in that After the intelligent image signal processor converts the electrical signal into a color image frame, the method further includes: In response to a second control instruction of the user, performing human figure detection based on the color image frame; In the case where a human-shaped target is detected in the color image frame, the direction and focal length of the lens with the preset aperture value are adjusted to track the human-shaped target.

5. The method according to claim 1, characterized in that The method further comprises: In response to a third control instruction from the user, cruise monitoring is performed within a preset time period at preset time intervals; the light beams corresponding to the various cruise angles of the cruise monitoring are different.

6. The method according to claim 1, characterized in that The monitoring terminal also includes at least two fill lights arranged in different directions; The method further comprises: In response to a fourth control instruction from the user, the brightness of a target fill light among the at least two fill lights is adjusted to a target brightness to change the intensity of the light.

7. The method according to claim 1, characterized in that The monitoring terminal also includes a microphone and a speaker; The method further comprises: In response to a fifth control instruction of the user, collecting a first sound source within a recording range of the monitoring terminal through the microphone, and sending a second reminder message based on the first sound source; and / or In response to a sixth control instruction of the user, a second sound source received by the monitoring terminal is played through the speaker; the second sound source is collected by a user terminal associated with the monitoring terminal.

8. An image processing device, characterized in that: Applied to a monitoring terminal, the monitoring terminal includes a lens with a preset aperture value, a sensor with a preset pixel size, and an intelligent image signal processor; the device includes: A light collection module, used for collecting light through the lens with the preset aperture value; A first conversion module, used for converting the light into an electrical signal through the sensor of the preset pixel size; A second conversion module, configured to convert the electrical signal into a color image frame through the intelligent image signal processor; The picture forming module is used to form a monitoring picture based on the color image frame.

9. An electronic device, characterized in that: include: A processor and a memory; the memory stores a computer program, and the processor implements the method steps of any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

Images