An image processing method, device, equipment and computer readable storage medium

Abstract

The application discloses an image processing method, device and equipment and a computer readable storage medium. The method comprises the following steps: acquiring an original image and a target region in the original image; performing image segmentation on the original image to obtain at least two image blocks; rearranging the at least two image blocks to obtain a background image; performing image fusion on the target region and the background image to obtain a target image; and the target image is used for training a preset detection network. By using the technical scheme, the original image can be subjected to directional data enhancement, and the training efficiency and training accuracy of the preset detection network can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image processing method and device, equipment and computer readable storage medium. BACKGROUND

[0002] In order to perform data enhancement on the image, the current technical solution includes: performing flip transformation, random cropping, color jitter, translation transformation, scale transformation, contrast transformation, noise disturbance, rotation transformation and reflection transformation, etc. on the image. These data enhancement solutions cannot perform directional data enhancement on the image, and thus cannot effectively improve the training efficiency and training accuracy of the preset detection model in the case of training the preset detection network using the data enhanced image, resulting in that the target detection capability of the target detection network obtained by training cannot be effectively improved. SUMMARY

[0003] The present application provides an image processing method, device, equipment and computer readable storage medium, which can at least perform directional data enhancement on the image, thereby improving the training efficiency and training accuracy of the preset detection network.

[0004] The present application provides an image processing method, which comprises:

[0005] obtaining an original image and a target region in the original image;

[0006] performing image segmentation on the original image to obtain at least two image blocks;

[0007] rearranging the at least two image blocks to obtain a background image;

[0008] performing image fusion on the target region and the background image to obtain a target image; wherein the target image is used to train a preset detection network.

[0009] In some optional embodiments, the rearranging the at least two image blocks to obtain a background image comprises:

[0010] determining target arrangement information of the at least two image blocks; wherein the target arrangement information represents target relative position information between the at least two image blocks, and the target relative position information is different from original relative position information of the at least two image blocks in the original image;

[0011] rearranging the at least two image blocks according to the target arrangement information to obtain the background image.

[0012] In some optional embodiments, the determining target arrangement information of the at least two image blocks comprises:

[0013] determining original position information;

[0014] randomly adjusting corresponding positions of the at least two image blocks in the original image based on the original position information, to obtain random position information;

[0015] determining the target arrangement information based on the random position information.

[0016] In some optional embodiments, the determining the target arrangement information of the at least two image blocks comprises:

[0017] obtaining preset position information;

[0018] determining the target arrangement information based on the preset position information.

[0019] In some optional embodiments, the image fusion of the target region and the background image to obtain a target image comprises:

[0020] determining target position information of the target region in the original image;

[0021] re-mapping the target region in the background image according to the target position information to obtain the target image.

[0022] In some optional embodiments, the image segmentation of the original image to obtain at least two image blocks comprises:

[0023] obtaining an aspect ratio of the original image;

[0024] determining an image segmentation size according to the aspect ratio;

[0025] segmenting the original image according to the image segmentation size to obtain the at least two image blocks.

[0026] In some optional embodiments, the method further comprises:

[0027] inputting the target image into the preset detection network to obtain a predicted class label and a confidence of the target image;

[0028] obtaining a target class label of the target image;

[0029] determining a target loss according to the target class label, the predicted class label and the confidence;

[0030] in a case where the target loss does not satisfy a preset condition, adjusting network parameters of the preset detection network according to the target loss, and updating the target loss based on the preset detection network after the network parameters are adjusted;

[0031] In a case where the target loss satisfies the preset condition, a preset detection network corresponding to the target loss satisfying the preset condition is taken as the target detection network.

[0032] In some optional embodiments, the method further includes:

[0033] obtaining a to-be-detected image, the to-be-detected image including a to-be-detected object;

[0034] inputting the to-be-detected image into a target detection network for target detection to obtain target object category information of the to-be-detected object.

[0035] The application further provides an image processing device, which includes:

[0036] a first obtaining module configured to obtain an original image and a target region in the original image;

[0037] an image segmentation module configured to perform image segmentation on the original image to obtain at least two image blocks;

[0038] a rearrangement module configured to rearrange the at least two image blocks to obtain a background image;

[0039] an image fusion module configured to perform image fusion on the target region and the background image to obtain a target image; wherein the target image is used for training a preset detection network.

[0040] In some optional embodiments, the rearrangement module includes:

[0041] a determining unit configured to determine target arrangement information of the at least two image blocks; wherein the target arrangement information represents target relative position information between the at least two image blocks, and the target relative position information is different from original relative position information of the at least two image blocks in the original image;

[0042] a rearrangement unit configured to rearrange the at least two image blocks according to the target arrangement information to obtain the background image.

[0043] In some optional embodiments, the determining unit includes:

[0044] a first determining subunit configured to determine original position information;

[0045] a random adjustment subunit configured to randomly adjust corresponding positions of the at least two image blocks in the original image based on the original position information to obtain the random position information;

[0046] The second determining sub-unit is configured to determine the target arrangement information based on the random position information.

[0047] In some optional embodiments, the determining unit comprises:

[0048] The obtaining sub-unit is configured to obtain preset position information.

[0049] The determining sub-unit is configured to determine the target arrangement information based on the preset position information.

[0050] In some optional embodiments, the image fusion module comprises:

[0051] The determining unit is configured to determine target position information of the target region in the original image.

[0052] The remapping unit is configured to remap the target region in the background image according to the target position information to obtain the target image.

[0053] In some optional embodiments, the image segmentation module comprises:

[0054] The obtaining unit is configured to obtain an aspect ratio of the original image.

[0055] The determining unit is configured to determine an image segmentation size according to the aspect ratio.

[0056] The image segmentation unit is configured to perform image segmentation on the original image according to the image segmentation size to obtain the at least two image blocks.

[0057] In some optional embodiments, the method further comprises:

[0058] The input module is configured to input the target image into the preset detection network to obtain a predicted class label and a confidence of the target image.

[0059] The second obtaining module is configured to obtain a target class label of the target image.

[0060] The first determining module is configured to determine a target loss according to the target class label, the predicted class label and the confidence.

[0061] The adjusting module is configured to, in a case where the target loss does not satisfy a preset condition, adjust network parameters of the preset detection network according to the target loss, and update the target loss based on the preset detection network after the network parameters are adjusted.

[0062] The second determining module is configured to, in a case where the target loss satisfies the preset condition, take a preset detection network corresponding to the case where the target loss satisfies the preset condition as the target detection network.

[0063] In some optional embodiments, the apparatus further includes:

[0064] a third obtaining module, configured to obtain a to-be-detected image, the to-be-detected image including a to-be-detected object;

[0065] a target detection module, configured to input the to-be-detected image into a target detection network for target detection, to obtain target object category information of the to-be-detected object.

[0066] The present application also provides an image processing device, including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described above.

[0067] The present application also provides a computer-readable storage medium, the storage medium storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the image processing method as described above.

[0068] The image processing method, apparatus, device and computer-readable storage medium provided by the present application have the following technical effects:

[0069] The present application can determine the region in the original image that needs to be directionally enhanced as the target region by obtaining the original image and the target region in the original image; can damage the image features of the original image by performing image segmentation on the original image to obtain at least two image blocks; can obtain a background image without complete image features by rearranging the at least two image blocks; can obtain a target image in which the target region is directionally enhanced by performing image fusion on the target region and the background image, thereby realizing directional (data) enhancement of the original image; and can effectively improve the training accuracy of the preset detection network, effectively reduce the number of training iterations, and improve the training efficiency of the preset detection network by using the target image for training of the preset detection network.

[0070] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0072] Figure 1 is an optional structural schematic diagram of a distributed system applied to a blockchain system provided by an embodiment of the present application.

[0073] Figure 2 is a flow schematic diagram of an image processing method provided by an embodiment of the present application.

[0074] Figure 3 is a flow schematic diagram of an image segmentation process provided by an embodiment of the present application.

[0075] Figure 4 is a flow schematic diagram of a preset detection network training process provided by an embodiment of the present application.

[0076] Figure 5 is a schematic diagram of an application scenario of an image processing method provided by an embodiment of the present application.

[0077] Figure 6 is a structural schematic diagram of an image processing device provided by an embodiment of the present application.

[0078] Figure 7 is a hardware structural block diagram of a server of an image processing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0080] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0081] The embodiments of the present application relate to a target detection system, which comprises a server and a terminal, the target detection system can realize the directional data enhancement of the original image through the server, obtain the target image, train the target detection network by using the target image, and realize the application of the target detection network through the terminal.

[0082] The target detection system related to the embodiments of the present application can be a distributed system formed by a client, a plurality of nodes (any form of computing device in an access network, such as a server, a user terminal) connected through network communication. The client can be deployed on the terminal.

[0083] Taking the distributed system as a blockchain system, referring to Figure 1 , Figure 1 is an optional structural schematic diagram of the distributed system 100 provided by the embodiments of the present application applied to the blockchain system, which is formed by a plurality of nodes (any form of computing device in an access network, such as a server, a user terminal) and a client, the nodes form a point-to-point (P2P, Peer To Peer) network, and the P2P protocol is an application layer protocol running on a transmission control protocol (TCP, Transmission Control Protocol) protocol. In the distributed system, any machine such as a server or a terminal can join to become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer and an application layer.

[0084] Referring to Figure 1 the functions of each node in the blockchain system, the functions involved include:

[0085] 1) Routing, a basic function of the node, used to support communication between nodes.

[0086] In addition to the routing function, the node can also have the following functions:

[0087] 2) Application, for deployment in the blockchain, to implement a specific business according to actual business needs, record the data related to the implementation function to form a record data, carry a digital signature in the record data to represent the source of the task data, send the record data to other nodes in the blockchain system, and other nodes add the record data to the temporary block when the source and integrity of the record data are verified successfully.

[0088] For example, the business implemented by the application includes:

[0089] 2.1) Wallet, for providing the function of conducting electronic currency transactions, including initiating transactions (i.e. sending transaction records of the current transaction to other nodes in the blockchain system, and other nodes store the record data of the transaction in the temporary block of the blockchain as a response to acknowledging the validity of the transaction after successful verification; of course, the wallet also supports querying the remaining electronic currency in the electronic currency address;

[0090] 2.2) Shared ledger, for providing the function of storing, querying and modifying account data, sending the record data of the operation on the account data to other nodes in the blockchain system, and other nodes store the record data in the temporary block as a response to acknowledging the validity of the account data after successful verification, and can also send a confirmation to the node that initiated the operation.

[0091] 2.3) Smart contract, a computerized agreement that can execute the terms of a contract, implemented by deploying code on the shared ledger for execution when certain conditions are met, the code is used to complete automated transactions according to actual business needs, such as querying the logistics status of the goods purchased by the buyer, and transferring the electronic currency of the buyer to the address of the merchant after the buyer signs for the goods; of course, the smart contract is not limited to executing contracts for transactions, but can also execute contracts for processing received information.

[0092] 3) Blockchain, including a series of blocks (Block) connected in chronological order according to the time of generation, once a new block is added to the blockchain, it will not be removed, and the block records the record data submitted by the nodes in the blockchain system.

[0093] In the embodiments of the present application, the training and application of the target detection network involve artificial intelligence technology, specifically the machine learning direction of artificial intelligence technology.

[0094] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0095] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0096] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0097] Specifically, in order to improve the data enhancement effect of the image, the application provides an image processing method to perform directional data enhancement on the original image to obtain a target image, and use the target image to train a preset detection network, thereby improving the training efficiency and training accuracy of the preset detection network.

[0098] The image processing method of the application is described below. The present specification provides method operation steps as described in the examples or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the examples is only one of the many execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the examples or drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0099] In a specific embodiment, as Figure 2As shown, the present application provides an image processing method, the method comprising:

[0100] S201: obtaining an original image and a target region in the original image.

[0101] In the embodiment of the present application, the original image can be an image that needs to be data enhanced.

[0102] In the embodiment of the present application, the original image can include a target object, and the type of the target object can include people, objects and scenes. Correspondingly, the target region can be the image region where the target object is located in the original image. It can be understood that the target region can be the region in the original image that needs to be directionally enhanced.

[0103] In one specific embodiment, the target region can be the minimum circumscribed rectangular region of the target object in the original image.

[0104] In an optional embodiment, the step of obtaining the target region comprises manually labeling the minimum circumscribed rectangle of the target object in the original image, and the computer recognizes the labeled original image to extract the target region from the original image.

[0105] In an optional embodiment, before obtaining the target region, the method further comprises determining one vertex of the original image as the origin, determining the horizontal edge where the origin is located as the x-axis and the vertical edge as the y-axis, and establishing a rectangular coordinate system. Correspondingly, the step of obtaining the target region comprises inputting the original image into a neural network based on deep learning to obtain the vertex coordinates of the minimum circumscribed rectangle of the target object, and obtaining the target region according to the vertex coordinates.

[0106] In actual application, using a target detection network to detect the target object in the original image may cause overfitting, specifically, the target object and the background region around the target object are recognized as the target object. In order to improve the recognition accuracy of the target detection network, the target region in the original image needs to be directionally enhanced, and the target image obtained after directional enhancement is used to train the preset detection network to obtain a target detection network with effectively improved recognition accuracy.

[0107] Considering that the overfitting phenomenon is to recognize the target object and the background region around the target object as the target object, the background region in the original image except the target object can be destroyed to achieve the purpose of directional enhancement. The target region can also be extracted, the original image can be destroyed, the background image can be determined based on the destroyed original image, the target region and the background image can be image fused to obtain a target image, and the target region in the target image can be directionally enhanced.

[0108] In the two schemes, the second scheme can more comprehensively destroy the image features of the original image, and obtain a background image that is less likely to overfit. Embodiments of the present application are based on the second scheme, and perform data enhancement on the original image.

[0109] In embodiments of the present application, by obtaining the region where the target is located in the original image, the region in the original image that needs to be directionally enhanced can be determined.

[0110] S203: performing image segmentation on the original image to obtain at least two image blocks.

[0111] In embodiments of the present application, the original image can be segmented into at least two image blocks of the same size. Specifically, the shape of the at least two image blocks can include a rectangle. To facilitate rearrangement of the at least two image blocks, a background image with the same image size as the original image is obtained.

[0112] In embodiments of the present application, the purpose of performing image segmentation on the original image is to destroy the image features of the original image. From the overall and local perspectives, the image features can include overall features and local features; from the type perspective, the image features include shape features, color features, texture features, and spatial relationship features.

[0113] In embodiments of the present application, image segmentation is performed on the original image, and it is necessary to ensure that the region where the target is located in the original image is destroyed. If the region where the target is located in the original image is not destroyed, it indicates that the image segmentation of the original image is not sufficient to destroy the local features of the original image, and thus the background image obtained by rearranging the at least two image blocks will have the region where the target is located. After image fusion is performed on the extracted region where the target is located and the background image, the target image obtained includes the extracted region where the target is located, and also includes the undamaged region where the target is located, which is not conducive to the identification of the preset detection network.

[0114] In a specific embodiment, the image segmentation size of the original image can be determined according to actual application requirements, to ensure that image segmentation is performed on the original image, and the region where the target is located in the original image can be destroyed.

[0115] S205: rearranging the at least two image blocks to obtain a background image.

[0116] In embodiments of the present application, the image size of the background image is the same as the image size of the original image.

[0117] It can be understood that the arrangement of the at least two image blocks in the background image is different from the arrangement of the at least two image blocks in the original image, and there is no complete image feature in the background image.

[0118] In this embodiment of the application, by rearranging at least two image blocks, a background image without complete image features can be obtained. Using this background image as the background of the target area can enhance the directionality of the target area.

[0119] S207: Perform image fusion on the region where the target is located and the background image to obtain a target image; wherein, the target image is used to train a preset detection network.

[0120] In this embodiment of the application, the target image can be a directionally enhanced image of the area where the target is located.

[0121] In this embodiment, the target image may include the area where the target is located, and the background area in the target image other than the area where the target is located does not have complete image features, which is beneficial for the preset detection network to identify the area where the target is located.

[0122] In this embodiment, training a preset detection network using the target image can improve the training efficiency and accuracy of the preset detection network, resulting in a target detection network with effectively improved recognition accuracy.

[0123] In S203, in order to segment the original image into at least two image blocks of the same size, such as Figure 3 The diagram shown is a flowchart illustrating an image segmentation process. (Refer to...) Figure 3 The step of segmenting the original image to obtain at least two image blocks includes:

[0124] S301: Obtain the aspect ratio of the original image.

[0125] It can be understood that the original image includes horizontal and vertical opposite sides. The horizontal opposite sides are of equal length, and the vertical opposite sides are of equal length.

[0126] In this embodiment of the application, the aspect ratio of the original image can be the ratio of the length of the horizontal opposite side to the length of the vertical opposite side.

[0127] S303: Determine the image segmentation size based on the aspect ratio.

[0128] In this embodiment, the image segmentation size may include a preset length and a preset width. Specifically, the ratio of the preset length to the preset width is equal to the aspect ratio.

[0129] S305: Based on the image segmentation size, perform image segmentation on the original image to obtain the at least two image blocks.

[0130] In the embodiments of the present application, the original image is segmented according to the image segmentation size, and each of the at least two image blocks obtained has a preset length and a preset width.

[0131] In a specific embodiment, the specific values of the preset length and the preset width are determined according to actual application requirements, so as to ensure that the original image is segmented according to the image segmentation size, and the image features of the original image are destroyed.

[0132] In the embodiments of the present application, the aspect ratio of the original image is obtained, the image segmentation size is determined according to the aspect ratio, and the original image is segmented according to the image segmentation size, so as to ensure that the at least two image blocks obtained are image blocks of the same size and the same shape, and the rearrangement of the at least two image blocks is facilitated.

[0133] In S205, in order to obtain a background image without complete image features, the rearrangement of the at least two image blocks to obtain the background image includes:

[0134] determining target arrangement information of the at least two image blocks; wherein the target arrangement information represents target relative position information between the at least two image blocks, and the target relative position information is different from original relative position information of the at least two image blocks in the original image;

[0135] rearranging the at least two image blocks according to the target arrangement information to obtain the background image.

[0136] In the embodiments of the present application, the target relative position information includes target relative positions of any two image blocks in the at least two image blocks, and the original relative position includes original relative positions of any two image blocks in the at least two image blocks.

[0137] In the embodiments of the present application, the positions of the at least two image blocks in the original image are arranged according to the target arrangement information, the rearrangement of the at least two image blocks is realized, and the background image is obtained.

[0138] In an optional embodiment, the target arrangement information of the at least two image blocks can be preset.

[0139] In an optional embodiment, the target arrangement information of the at least two image blocks can be determined randomly.

[0140] In the case of presetting the target arrangement information of the at least two image blocks, the determination of the target arrangement information of the at least two image blocks includes:

[0141] obtaining preset position information;

[0142] Determine the target arrangement information of the at least two image blocks based on the preset position information.

[0143] In the embodiments of the present application, the preset position information can be the positions corresponding to the at least two image blocks.

[0144] In the embodiments of the present application, based on the positions corresponding to the at least two image blocks, the target relative positions of any two image blocks in the at least two image blocks are determined, which can improve the determination efficiency of the target arrangement information.

[0145] In the case of randomly determining the target arrangement information of the at least two image blocks, the determining the target arrangement information of the at least two image blocks includes:

[0146] Determine the original position information;

[0147] Randomly adjust the positions corresponding to the at least two image blocks in the original image based on the original position information to obtain the random position information;

[0148] Determine the target arrangement information based on the random position information.

[0149] In the embodiments of the present application, the original position information includes the original positions corresponding to the at least two image blocks in the original image.

[0150] In the embodiments of the present application, the random position information includes the positions corresponding to the at least two image blocks after the random adjustment of the at least two image blocks.

[0151] In one specific embodiment, the positions corresponding to any two image blocks in the original image can be randomly exchanged based on the original positions corresponding to the at least two image blocks in the original image to realize the random adjustment of the at least two image blocks.

[0152] In the embodiments of the present application, the random determination of the target arrangement information of the at least two image blocks can improve the disorder of the target arrangement information and is beneficial to obtaining the original image with higher image feature incompleteness of the target region.

[0153] It can be understood that since the target region in the original image is destroyed in the process of obtaining the background image, image fusion needs to be performed on the target region and the background image to obtain the target image. In order to make the image features of the target region in the target image closer to the image features of the target region in the original image.

[0154] In S207, the image fusion of the target region and the background image to obtain the target image includes:

[0155] Determine the target position information of the target region in the original image;

[0156] According to the target position information, the target region is remapped to the background image to obtain the target image.

[0157] In the embodiments of the present application, the target position information can include coordinate position information of the target region in the original image. The target position information can also include pixel position information of the target region in the original image.

[0158] Specifically, in the case where the target region is the minimum circumscribed rectangle of the target object in the original image, the above-mentioned coordinate position information can include the vertex coordinates of the minimum circumscribed rectangle, and can include the pixel coordinates of the minimum circumscribed rectangle.

[0159] According to the vertex coordinates or the pixel coordinates of the minimum circumscribed rectangle, the minimum circumscribed rectangle is remapped to the background image to obtain the target image.

[0160] In the embodiments of the present application, by remapping the target region to the background image, the image features of the target region in the target image can be made closer to the image features of the target region in the original image, so as to realize the integrity of the image features of the target region in the target image.

[0161] In one specific embodiment, as shown in Figure 4 the flowchart of a training process of a preset detection network. Specifically, the method further includes:

[0162] S401: inputting the target image into the preset detection network to obtain a predicted class label and a confidence of the target image.

[0163] In the embodiments of the present application, the preset detection network can be a neural network based on deep learning which needs to be trained.

[0164] In the embodiments of the present application, the target image includes a target region, and the target region includes a target object. Correspondingly, the preset class label can represent the class of the target object predicted by the preset detection network. Correspondingly, the confidence can be the accuracy of the preset class label predicted by the prediction network.

[0165] S403: obtaining a target class label of the target image.

[0166] In the embodiments of the present application, the target class label can represent the real class of the target object.

[0167] S405: determining a target loss according to the target class label, the predicted class label and the confidence.

[0168] In the embodiments of the present application, the target loss can be calculated through a cross-entropy loss function. Specifically, the formula of the cross-entropy loss function is:

[0169]

[0170] wherein N is the number of pixels in the target image, i represents the i-th pixel in the current iteration, L i is the target sub-loss corresponding to the pixel in the current iteration, p i is the confidence corresponding to the i-th pixel, y i is the predicted class label corresponding to the i-th pixel, and the value of 1 indicates that the predicted class label is the same as the target class label, and the value of 0 indicates that the predicted class label is different from the target class label.

[0171] S407: In the case where the target loss does not meet the preset condition, adjusting the network parameters of the preset detection network according to the target loss, and updating the target loss based on the preset detection network after adjusting the network parameters.

[0172] In the embodiments of the present application, the preset detection network includes an input layer, an intermediate layer and an output layer. Correspondingly, the network parameters of the prediction detection network include input layer parameters, intermediate layer parameters and output layer parameters.

[0173] In an optional embodiment, the preset condition is met, including that the target loss is lower than a preset first threshold, and the first threshold can be determined according to actual application requirements.

[0174] In an optional embodiment, the preset condition is met, including that the number of iterations in the training stage reaches a preset second threshold, and the second threshold can be determined according to actual application requirements.

[0175] S409: In the case where the target loss meets the preset condition, the preset detection network corresponding to the case where the target loss meets the preset condition is taken as the target detection network.

[0176] In the embodiments of the present application, the target image is used to train the prediction detection network, and since the target image includes the directional enhanced target region and is easy to be recognized by the preset detection network, the number of training iterations of the prediction detection network can be reduced, the training efficiency and accuracy can be improved, and the target detection network obtained by training can also have better recognition ability.

[0177] In an optional embodiment, in order to realize the application of the target detection network, the above method further includes:

[0178] obtaining a to-be-detected image, wherein the to-be-detected image includes a to-be-detected object;

[0179] The image to be detected is input into a target detection network for target detection to obtain the target object category information of the object to be detected.

[0180] In this embodiment, the target object category information may include the target object label and target confidence score of the object to be detected. The target object label represents the object category predicted by the target detection network, and the object category type may include people, objects, and scenes. The target confidence score represents the accuracy of the object category of the object to be detected.

[0181] In one specific embodiment, the object detection network can predict and perform object detection for objects of different object categories, each with a different confidence threshold. When the target confidence score is greater than or equal to the confidence threshold corresponding to the object category of the object to be detected, the target object label can be determined to be the correct label.

[0182] In this embodiment, confidence thresholds for different object categories can be set according to actual application requirements. Since the target detection network obtained in this embodiment has significantly improved recognition capabilities compared to conventional target detection networks, higher confidence thresholds can be set for different object categories. Using higher confidence thresholds to measure whether the target confidence output by the target detection network meets the standard can greatly improve the recall and accuracy of the target detection network. Experimental data shows that training the preset detection network with target images obtained using the image processing method provided in this embodiment can improve the recall and accuracy of the target detection network by 150%.

[0183] like Figure 5 The diagram shown illustrates an application scenario of image processing provided in this application. (Refer to...) Figure 5 , Figure 5 The document sequentially displays the original image, at least two image blocks obtained by image segmentation of the original image, a background image obtained by rearranging the at least two image blocks, and a target image obtained by remapping the target region onto the background image.

[0184] The present application can determine the region needing directional enhancement in the original image as the target region by acquiring the original image and the target region in the original image, can damage the image features of the original image by image segmentation of the original image to obtain at least two image blocks, can obtain the background image without complete image features by rearranging the at least two image blocks, can obtain the target image with directional enhancement of the target region by image fusion of the target region and the background image, so as to realize directional enhancement of the original image, can effectively improve the training accuracy of the preset detection network by using the target image to train the preset detection network, can also effectively reduce the training iteration times and improve the training efficiency of the preset detection network. The recognition ability of the target detection network obtained by training the preset detection network using the target image can also be effectively improved. In addition, the image processing method provided by the present application is simple and easy to implement, does not need to occupy more CPU resources, is also suitable for online real-time image data enhancement scene, can realize real-time data enhancement of the original image, and can input the obtained target image into the preset detection network for training of the preset detection network.

[0185] In addition, the image processing method provided by the embodiment of the present application does not need to consider the format of the image, and is suitable for processing of images of various formats.

[0186] As shown in Figure 6 , the embodiment of the present application further provides a structure diagram of an image processing device 600. Referring to Figure 6 , the above device comprises:

[0187] The first acquisition module 601 is configured to acquire an original image and a target region in the original image.

[0188] The image segmentation module 603 is configured to perform image segmentation on the original image to obtain at least two image blocks.

[0189] The rearrangement module 605 is configured to rearrange the at least two image blocks to obtain a background image.

[0190] The image fusion module 607 is configured to perform image fusion on the target region and the background image to obtain a target image; wherein the target image is used to train a preset detection network.

[0191] In some embodiments, the above rearrangement module 605 comprises:

[0192] The determination unit is configured to determine target arrangement information of the at least two image blocks; wherein the target arrangement information represents target relative position information between the at least two image blocks, and the target relative position information is different from original relative position information of the at least two image blocks in the original image.

[0193] a rearrangement unit, configured to rearrange the at least two image blocks according to the target arrangement information, to obtain the background image.

[0194] In some embodiments, the determining unit includes:

[0195] a first determining sub-unit, configured to determine original position information;

[0196] a random adjustment sub-unit, configured to randomly adjust corresponding positions of the at least two image blocks in the original image based on the original position information, to obtain random position information;

[0197] a second determining sub-unit, configured to determine the target arrangement information based on the random position information.

[0198] In some embodiments, the determining unit includes:

[0199] an obtaining sub-unit, configured to obtain preset position information;

[0200] a determining sub-unit, configured to determine the target arrangement information based on the preset position information.

[0201] In some embodiments, the image fusion module 607 includes:

[0202] a determining unit, configured to determine target position information of the target region in the original image;

[0203] a remapping unit, configured to remap the target region in the background image according to the target position information, to obtain the target image.

[0204] In some embodiments, the image segmentation module 603 includes:

[0205] an obtaining unit, configured to obtain an aspect ratio of the original image;

[0206] a determining unit, configured to determine an image segmentation size according to the aspect ratio;

[0207] an image segmentation unit, configured to perform image segmentation on the original image according to the image segmentation size, to obtain the at least two image blocks.

[0208] In some embodiments, the apparatus further includes:

[0209] an input module, configured to input the target image into the preset detection network, to obtain a predicted class label and a confidence of the target image;

[0210] The second obtaining module is configured to obtain a target category label of the target image.

[0211] The first determining module is configured to determine a target loss according to the target category label, the predicted category label and the confidence.

[0212] The adjusting module is configured to, in a case where the target loss does not satisfy a preset condition, adjust a network parameter of the preset detection network according to the target loss, and update the target loss based on the preset detection network after the network parameter is adjusted.

[0213] The second determining module is configured to, in a case where the target loss satisfies the preset condition, take a preset detection network corresponding to the case where the target loss satisfies the preset condition as the target detection network.

[0214] In some embodiments, the apparatus further includes:

[0215] The third obtaining module is configured to obtain a to-be-detected image, the to-be-detected image including a to-be-detected object.

[0216] The target detection module is configured to input the to-be-detected image into a target detection network for target detection, to obtain target object category information of the to-be-detected object.

[0217] The apparatus and the method in the apparatus embodiment are based on the same inventive concept.

[0218] Embodiments of the present application further provide an image processing device, which includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method provided by the above method embodiments.

[0219] The device and the method in the device embodiment are based on the same inventive concept.

[0220] Embodiments of the present application further provide a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the image processing method provided by the above method embodiments.

[0221] The storage medium and the method in the computer-readable storage medium embodiment are based on the same inventive concept.

[0222] The application further provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementation manners.

[0223] The image processing server provided in the embodiments of the application comprises a processor and a memory. The memory stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image processing method provided in the method embodiments.

[0224] The memory can be used to store software programs and modules. The processor executes various functions and image processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by functions and the like. The data storage area can store data created according to the use of the device and the like. In addition, the memory can include a high-speed random access memory and can also include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0225] The method embodiments provided in the embodiments of the application can be executed in a mobile terminal, a computer terminal, a server or similar computing devices. Taking the case of running on a server as an example, Figure 7 is a hardware structure block diagram of a server of an image processing method provided in the embodiments of the application. As shown in Figure 7As shown, the server 700 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 710 (CPUs 710 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 730 for storing data, and one or more storage media 720 (e.g., one or more mass storage devices) for storing application programs 723 or data 722. The memory 730 and storage media 720 may be temporary or persistent storage. The program stored in the storage media 720 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 710 may be configured to communicate with the storage media 720 and execute the series of instruction operations stored in the storage media 720 on the server 700. Server 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input / output interfaces 740, and one or more operating systems 721, such as Windows Server™, Mac OS x™, Unix™, Linux™, FreeBSD™, etc.

[0226] The input / output interface 740 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 700. In one example, the input / output interface 740 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 740 may be a radio frequency (RF) module for wireless communication with the Internet.

[0227] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 700 may also include... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0228] The embodiments of this application also provide a storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set or instruction set related to implementing an image processing method in the method embodiments. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the image processing method provided in the above method embodiments.

[0229] Optionally, in the embodiment, the storage medium can be located in at least one of the plurality of network servers of the computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media capable of storing program codes.

[0230] It can be seen from the embodiments of the image processing method, device, equipment and computer readable storage medium provided by the present application that the target region in the original image can be determined as the target region in need of directional enhancement by acquiring the original image and the target region in the original image; the image features of the original image can be damaged by performing image segmentation on the original image to obtain at least two image blocks; the background image without complete image features can be obtained by rearranging the at least two image blocks; the target image in which the target region is directionally enhanced can be obtained by performing image fusion on the target region and the background image, so as to realize directional enhancement of the original image; the training accuracy of the preset detection network can be effectively improved, and the training iteration number can be effectively reduced and the training efficiency of the preset detection network can be improved by using the target image for training of the preset detection network.

[0231] It should be noted that the above-mentioned embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0232] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, the device embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments.

[0233] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed to relevant hardware by program. The program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0234] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire the original image and the region where the target is located in the original image; The original image is segmented to obtain at least two image blocks, wherein the image segmentation is used to at least destroy the target region of the original image; The at least two image blocks are rearranged to obtain a background image; Determine the target location information of the region where the target is located in the original image; Based on the target location information, the region where the target is located is remapped onto the background image to obtain the target image; wherein, the target image is used to train a preset detection network.

2. The method according to claim 1, characterized in that, The rearrangement of the at least two image blocks to obtain the background image includes: Determine the target arrangement information of the at least two image blocks; wherein the target arrangement information characterizes the target relative position information between the at least two image blocks, and the target relative position information is different from the original relative position information of the at least two image blocks in the original image; Based on the target arrangement information, the at least two image blocks are rearranged to obtain the background image.

3. The method according to claim 2, characterized in that, Determining the target arrangement information of the at least two image blocks includes: Determine the original location information; Based on the original position information, the positions of the at least two image blocks in the original image are randomly adjusted to obtain random position information; Based on the random position information, the target arrangement information is determined.

4. The method according to claim 2, characterized in that, Determining the target arrangement information of the at least two image blocks includes: Obtain preset location information; Based on the preset position information, the target arrangement information is determined.

5. The method according to any one of claims 1 to 4, characterized in that, The target region is the image region in the original image where the target object is located.

6. The method according to any one of claims 1 to 4, characterized in that, The step of segmenting the original image to obtain at least two image blocks includes: Obtain the aspect ratio of the original image; Determine the image segmentation size based on the aspect ratio; Based on the image segmentation size, the original image is segmented to obtain at least two image blocks.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The target image is input into the preset detection network to obtain the predicted category label and confidence level of the target image; Obtain the target category label of the target image; The target loss is determined based on the target category label, the predicted category label, and the confidence level. If the target loss does not meet the preset conditions, the network parameters of the preset detection network are adjusted according to the target loss, and the target loss is updated based on the preset detection network after the network parameters are adjusted. If the target loss satisfies the preset conditions, the preset detection network corresponding to the target loss satisfying the preset conditions shall be used as the target detection network.

8. The method according to claim 7, characterized in that, The method further includes: Acquire an image to be detected, wherein the image to be detected includes an object to be detected; The image to be detected is input into a target detection network for target detection to obtain the target object category information of the object to be detected.

9. An image processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the original image and the region where the target is located in the original image; An image segmentation module is used to segment the original image to obtain at least two image blocks, wherein the image segmentation is used to at least destroy the target region of the original image; A rearrangement module is used to rearrange the at least two image blocks to obtain a background image; An image fusion module is used to fuse the target region and the background image to obtain a target image; wherein the target image is used to train a preset detection network; The image fusion module includes: The determining unit is used to determine the target location information of the region where the target is located in the original image; The remapping unit is used to remap the area where the target is located onto the background image based on the target location information to obtain the target image.

10. The image processing apparatus according to claim 9, characterized in that, The rearrangement module includes: A determining unit is configured to determine target arrangement information of the at least two image blocks; wherein the target arrangement information characterizes the target relative position information between the at least two image blocks, and the target relative position information is different from the original relative position information of the at least two image blocks in the original image; The rearrangement unit is used to rearrange the at least two image blocks according to the target arrangement information to obtain the background image.

11. The image processing apparatus according to claim 10, characterized in that, The determining unit includes: The first determining subunit is used to determine the original location information; The random adjustment subunit is used to randomly adjust the corresponding positions of the at least two image blocks in the original image based on the original position information to obtain random position information; The second determining subunit is used to determine the target arrangement information based on the random position information.

12. The image processing apparatus according to claim 10, characterized in that, The determining unit includes: Acquire sub-units to obtain preset position information; A subunit is defined for determining the target arrangement information based on the preset position information.

13. The image processing apparatus according to any one of claims 9-12, characterized in that, The image segmentation module includes: An acquisition unit is used to acquire the aspect ratio of the original image; The determining unit is used to determine the image segmentation size based on the aspect ratio; An image segmentation unit is used to segment the original image according to the image segmentation size to obtain the at least two image blocks.

14. The image processing apparatus according to any one of claims 9-12, characterized in that, Also includes: The input module is used to input the target image into the preset detection network to obtain the predicted category label and confidence level of the target image; The second acquisition module is used to acquire the target category label of the target image; The first determining module is used to determine the target loss based on the target category label, the predicted category label, and the confidence level; An adjustment module is used to adjust the network parameters of the preset detection network according to the target loss when the target loss does not meet the preset conditions, and to update the target loss based on the preset detection network after adjusting the network parameters. The second determining module is used to, when the target loss satisfies the preset condition, use the preset detection network corresponding to the target loss satisfying the preset condition as the target detection network.

15. The image processing apparatus according to claim 14, characterized in that, The device further includes: The third acquisition module is used to acquire the image to be detected, wherein the image to be detected includes the object to be detected; The target detection module is used to input the image to be detected into the target detection network for target detection, and obtain the target object category information of the object to be detected.

16. An image processing apparatus, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 8.

17. A computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image processing method as claimed in any one of claims 1 to 8.

Images