Structured recognition method and device of image, electronic equipment and readable storage medium

Abstract

The present disclosure provides a structured recognition method and device of an image, an electronic device and a readable storage medium. The method comprises: acquiring first structured data corresponding to a first image; performing enhancement processing on the first image according to the first structured data to obtain a second image; acquiring second structured data corresponding to the second image; and determining the second image and the second structured data as a structured recognition result.

Description

Technical Field

[0001] This disclosure relates to the field of recognition technology, and more specifically, to a method, apparatus, electronic device, and readable storage medium for structured image recognition. Background Technology

[0002] Currently, artificial intelligence technology is widely used in video enhancement, video restoration, intelligent editing, and structured image recognition, significantly improving the efficiency of online audiovisual content production. Structured image recognition can be used for the classification, labeling, and rapid inspection of image and video materials, which is an important foundation for technologies such as the production of massive amounts of material, video fingerprinting, and intelligent editing.

[0003] Because online audiovisual media platforms have high requirements for the real-time nature of media content, improving the accuracy of structured image recognition has become an urgent problem to be solved. Summary of the Invention

[0004] One object of this disclosure is to provide a structured image recognition method to improve the structured image recognition rate.

[0005] According to a first aspect of the present disclosure, a method for structured image recognition is provided, the method comprising:

[0006] Obtain the first structured data corresponding to the first image;

[0007] The first image is enhanced based on the first structured data to obtain the second image;

[0008] Obtain the second structured data corresponding to the second image;

[0009] The second image and the second structured data are identified as the structured recognition result.

[0010] Optionally, the first structured data includes at least: the area and coordinates of the first target region where at least one target object in the first image is located;

[0011] The step of enhancing the first image based on the first structured data to obtain the second image includes:

[0012] The region to be enhanced in the first image is determined based on the area and coordinates of at least one of the first target regions.

[0013] The image within the area to be enhanced is then enhanced to obtain a second image.

[0014] Optionally, determining the region to be enhanced in the first image based on the area and coordinates of the first target region includes:

[0015] In the case where the first target regions overlap, the region to be enhanced includes all the overlapping first target regions, and the area of ​​the region to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all the overlapping first target regions to the area of ​​the region to be enhanced is maximized.

[0016] Optionally, determining the region to be enhanced in the first image based on the area and coordinates of the first target region includes:

[0017] If there is no overlap in the first target region, the first target region is determined as the region to be enhanced.

[0018] Optionally, the enhancement process performed on the image within the area to be enhanced to obtain a second image includes:

[0019] The image within the area to be enhanced is sequentially subjected to sharpness enhancement processing and high dynamic range enhancement processing to obtain the second image.

[0020] Optionally, the second structured data includes at least: the area and coordinates of the second target region where at least one target object is located in the second image, and the label and label confidence level corresponding to the at least one target object.

[0021] According to a second aspect of the present disclosure, a structured image recognition apparatus is provided, the apparatus comprising:

[0022] The acquisition module is used to acquire the first structured data corresponding to the first image;

[0023] An enhancement module is used to enhance the first image based on the first structured data to obtain a second image;

[0024] The acquisition module is also used to acquire second structured data corresponding to the second image;

[0025] The determination module is used to determine the second image and the second structured data as a structured recognition result.

[0026] Optionally, the first structured data includes at least: the area and coordinates of the first target region where at least one target object in the first image is located;

[0027] The enhancement module is specifically used for:

[0028] The region to be enhanced in the first image is determined based on the area and coordinates of at least one of the first target regions.

[0029] The image within the area to be enhanced is then enhanced to obtain a second image.

[0030] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0031] Memory is used to store executable computer instructions;

[0032] A processor is configured to execute, under the control of the executable computer instructions, the image structure recognition method according to any one of the first aspects of the present disclosure.

[0033] According to a fourth aspect of the present disclosure, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the structured image recognition method as described in any one of the first aspects of the present disclosure.

[0034] One beneficial effect of this embodiment is that by acquiring first structured data corresponding to a first image; enhancing the first image based on the first structured data to obtain a second image; acquiring second structured data corresponding to the second image; and determining the second image and the second structured data as the structured recognition result, the accuracy of structured recognition of the image can be effectively improved without reducing processing efficiency by enhancing the region where the target object is located in the image.

[0035] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0037] Figure 1 This is a block diagram of the hardware configuration of an electronic device 1000 according to an embodiment of the present disclosure;

[0038] Figure 2 A schematic flowchart of a structured image recognition method according to an embodiment of the present disclosure is shown;

[0039] Figure 3 A schematic block diagram of an image structure recognition device according to an embodiment of the present disclosure;

[0040] Figure 4A schematic block diagram of another image structure recognition device according to an embodiment of the present disclosure;

[0041] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment. Detailed Implementation

[0042] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0043] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0044] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0045] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0047] <Hardware Configuration>

[0048] Figure 1 This is a block diagram of the hardware configuration of an electronic device 1000 according to an embodiment of the present disclosure.

[0049] like Figure 1 As shown, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a microphone 1700, and a speaker 1800.

[0050] Processor 1100 may be a mobile processor, including but not limited to a central processing unit (CPU) and a microprocessor (MCU). Memory 1200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. Interface device 1300 includes, for example, various bus interfaces, such as serial bus interfaces (including USB interfaces) and parallel bus interfaces. Communication device 1400 may include short-range communication devices, such as any device performing short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. Communication device 1400 may also include long-range communication devices, such as any device performing WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. Display device 1500 is, for example, an LCD screen, an LED screen, or a touch screen. Input device 1600 includes, for example, a touch screen. Microphone 1700 can be used to input voice information. Speaker 1800 can be used to output voice information.

[0051] In this embodiment, the memory 1200 of the electronic device 1000 is used to store instructions for controlling the processor 1100 to operate in order to implement or support the implementation of a structured image recognition method according to any embodiment. Those skilled in the art can design instructions based on the schemes disclosed in this specification. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0052] Those skilled in the art should understand that, although in Figure 1 The present invention illustrates a plurality of devices of an electronic device 1000; however, the electronic device 1000 of the present invention may involve only some of the devices, or may include other devices, and is not limited herein.

[0053] <Method Implementation>

[0054] Figure 2 A schematic flowchart illustrating a structured image recognition method according to an embodiment of this disclosure is shown. This structured image recognition method can be, for example, by... Figure 1 The electronic device 1000 shown is implemented.

[0055] like Figure 2 As shown, the structured recognition method for images provided in this embodiment may include the following steps 2100-2400.

[0056] Step 2100: Obtain the first structured data corresponding to the first image.

[0057] The first image can be a picture or a video frame, without specific limitation. Specifically, the electronic device 1000 can perform structuring processing on the first image to obtain first structured data corresponding to the first image.

[0058] Specifically, the first structured data may include at least: the area and coordinates of a first target region where at least one target object is located in the first image. For example, the target object may be a person, object, animal, or other object contained in the first image. In one example, a target detection algorithm can be used to determine the first target region where at least one target object is located in the first image, and to determine the area and coordinates of the first target region.

[0059] It is understood that when the first image contains multiple target objects, the first structured data may include the area and coordinates of multiple first target regions.

[0060] Furthermore, the first structured data may also include the tags of the target object and the corresponding tag confidence levels. The tags of the target object may be categories of the target object, and the tag confidence level refers to the probability that the target object belongs to that category.

[0061] For example, assuming the target object is a puppy, the tag for the target object can be "animal" or "dog". Correspondingly, the credibility of the tag can be the probability that the target object is "animal" or "dog".

[0062] Step 2200: Enhance the first image based on the first structured data to obtain the second image.

[0063] In this step, the electronic device 1000 can determine the region to be enhanced in the first image based on the area and coordinates of at least one first target region; and perform enhancement processing on the image within the region to be enhanced to obtain a second image.

[0064] Specifically, when determining the region to be enhanced, the electronic device 1000 may first determine whether at least one of the first target regions overlaps, and then determine the region to be enhanced based on the determination result.

[0065] For example, when the first target regions overlap, the region to be enhanced includes all the overlapping first target regions, and the area of ​​the region to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all the overlapping first target regions to the area of ​​the region to be enhanced is maximized.

[0066] That is, when the determination result is that at least one of the first target regions overlaps, the electronic device 1000 determines the area containing a group of overlapping first target regions as the region to be enhanced, and determines the region to be enhanced based on the principle of maximizing the ratio of the area of ​​the overlapping group of first target regions to the area of ​​the region to be enhanced. For example, if three first target regions overlap, the region to be enhanced needs to include these three first target regions, and the ratio of the sum of the areas of these three first target regions to the area of ​​the region to be enhanced satisfies the principle of maximizing the ratio.

[0067] For example, if there is no overlap between the first target regions, the first target region is determined as the region to be enhanced. That is, when the determination result is that at least one of the first target regions does not overlap, the electronic device 1000 determines each first target region as one region to be enhanced. In other words, if the number of first target regions is 3, then the number of regions to be enhanced is also 3.

[0068] In this embodiment, when the electronic device 1000 enhances the image within the area to be enhanced, it can sequentially perform sharpness enhancement processing and high dynamic range enhancement processing on the image within the area to be enhanced to obtain the second image. It should be noted that the enhancement algorithm used in this enhancement processing can be set according to actual needs and is not specifically limited here.

[0069] Step 2300: Obtain the second structured data corresponding to the second image.

[0070] After obtaining a second image with localized enhancements, the electronic device 1000 performs further structuring processing on the second image to obtain second structured data. The second structured data includes at least: the area and coordinates of a second target region containing at least one target object in the second image, and the label and label confidence level corresponding to the at least one target object.

[0071] Step 2400: The second image and the second structured data are determined as the structured recognition result.

[0072] In this embodiment, the second image with local region enhancement and the second structured data corresponding to the second image are output as the structured recognition result. Compared with image enhancement processing of the entire image, the structured recognition accuracy of the image can be effectively improved while ensuring processing efficiency.

[0073] According to this embodiment, first structured data corresponding to a first image is acquired; the first image is enhanced based on the first structured data to obtain a second image; second structured data corresponding to the second image is acquired; and the second image and the second structured data are determined as the structured recognition result. In this way, by enhancing the region where the target object is located in the image, the accuracy of structured recognition of the image can be effectively improved without reducing processing efficiency.

[0074] <Device Embodiment>

[0075] This disclosure provides an image structure recognition device, installed in an electronic device, such as... Figure 3 As shown, the structured recognition device 3000 for the image may include: an acquisition module 3100, an enhancement module 3200, and a determination module 3300.

[0076] The acquisition module 3100 is used to acquire the first structured data corresponding to the first image.

[0077] The enhancement module 3200 is used to enhance the first image based on the first structured data to obtain a second image.

[0078] The acquisition module 3100 is also used to acquire second structured data corresponding to the second image.

[0079] The determination module 3300 is used to determine the second image and the second structured data as a structured recognition result.

[0080] In one embodiment, the first structured data includes at least: the area and coordinates of a first target region where at least one target object in the first image is located; the enhancement module 3200 is specifically used to: determine the region to be enhanced in the first image based on the area and coordinates of at least one first target region; and perform enhancement processing on the image within the region to be enhanced to obtain a second image.

[0081] In one embodiment, the enhancement module 3200 is specifically configured to, when the first target regions overlap, include all overlapping first target regions, and the area of ​​the area to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all overlapping first target regions to the area of ​​the area to be enhanced is maximized. When the first target regions do not overlap, the first target region is determined as the area to be enhanced.

[0082] In one embodiment, the enhancement module 3200 is specifically used to sequentially perform sharpness enhancement processing and high dynamic range enhancement processing on the image within the area to be enhanced, thereby obtaining the second image.

[0083] In one embodiment, the second structured data includes at least: the area and coordinates of the second target region where at least one target object is located in the second image, and the label and label confidence level corresponding to the at least one target object.

[0084] Optionally, in a specific implementation, such as Figure 4 As shown, the image structure recognition device 4000 of this embodiment may further include: a structure processing module 4100, an enhancement strategy manager 4200, and an image enhancement module 4300.

[0085] The structured processing module 4100 receives a first image and performs target detection processing on the input first image to obtain first structured data. The first structured data may include the area and coordinates of a first target region containing at least one target object in the first image, the label of the target object, and the corresponding label confidence level. The first structured data is then sent to the enhancement strategy manager 4200.

[0086] The enhancement strategy manager 4200 receives the first structured data and determines the region to be enhanced in the first image based on the area of ​​the first target region, the region coordinates, the label of the target object, and the label confidence level.

[0087] Specifically, the strategy for determining the region to be enhanced includes: when the first target regions overlap, the region to be enhanced includes all the overlapping first target regions, and the area of ​​the region to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all the overlapping first target regions to the area of ​​the region to be enhanced is maximized.

[0088] If there is no overlap in the first target region, the first target region is determined as the region to be enhanced.

[0089] The image enhancement module 4300 receives the area to be enhanced from the enhancement strategy manager 4200, and sequentially performs sharpness enhancement processing and high dynamic range enhancement processing on the image within the area to be enhanced to obtain a second image.

[0090] The second image is then input to the structured processing module 4100, which performs structured processing on the second image to obtain second structured data. The second structured data and the second image are then simultaneously output as the structured recognition result.

[0091] The image structure recognition device in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0092] <Equipment Example>

[0093] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment. For example... Figure 5 As shown, the electronic device 5000 may include: a memory 5100 for storing executable computer instructions; and a processor 5200 for executing the image structure recognition method described in the above embodiments under the control of the executable computer instructions.

[0094] In another embodiment, the electronic device 5000 may include a structured recognition device 3000 for the above images.

[0095] In one embodiment, each module of the structured recognition device 3000 for the above images can be implemented by the processor 5200 running computer instructions stored in the memory 5100.

[0096] Computer-readable storage media

[0097] This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform the detection method for a head-mounted display device provided in this disclosure.

[0098] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0099] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0100] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0101] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0102] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0104] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0106] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A structured image recognition method, characterized in that, The method includes: Obtain first structured data corresponding to the first image; the first structured data includes at least: the area and coordinates of the first target region where at least one target object in the first image is located; The region to be enhanced in the first image is determined based on the area and coordinates of at least one of the first target regions. The image within the area to be enhanced is enhanced to obtain a second image; Obtain the second structured data corresponding to the second image; The second image and the second structured data are identified as the structured recognition result; The step of determining the region to be enhanced of the first image based on the area and coordinates of at least one first target region includes: when the first target regions overlap, the region to be enhanced includes all the overlapping first target regions, and the area of ​​the region to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all the overlapping first target regions to the area of ​​the region to be enhanced is maximized.

2. The method according to claim 1, characterized in that, Determining the region to be enhanced in the first image based on the area and coordinates of the first target region includes: If there is no overlap in the first target region, the first target region is determined as the region to be enhanced.

3. The method according to claim 1, characterized in that, The enhancement process for the image within the area to be enhanced, to obtain a second image, includes: The image within the area to be enhanced is sequentially subjected to sharpness enhancement processing and high dynamic range enhancement processing to obtain the second image.

4. The method according to claim 1, characterized in that, The second structured data includes at least: the area and coordinates of the second target region where at least one target object is located in the second image, and the label and label confidence level corresponding to the at least one target object.

5. A structured image recognition device, characterized in that, The device includes: The acquisition module is used to acquire first structured data corresponding to the first image; the first structured data includes at least: the area and coordinates of the first target region where at least one target object in the first image is located; An enhancement module is used to determine the region to be enhanced in the first image based on the area and coordinates of at least one first target region; and to perform enhancement processing on the image within the region to be enhanced to obtain a second image. The acquisition module is also used to acquire second structured data corresponding to the second image; The determining module is used to determine the second image and the second structured data as a structured recognition result; Specifically, the enhancement module is used for: In the case where the first target regions overlap, the region to be enhanced includes all the overlapping first target regions, and the area of ​​the region to be enhanced satisfies a preset condition; the preset condition is that the ratio of the area of ​​all the overlapping first target regions to the area of ​​the region to be enhanced is maximized.

6. An electronic device, characterized in that, include: Memory is used to store executable computer instructions; A processor, configured to execute the structured image recognition method according to any one of claims 1-4, under the control of the executable computer instructions.

7. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image structured recognition method as described in any one of claims 1 to 4.

Images