Two-dimensional code image segmentation method and device, electronic equipment and storage medium

Abstract

The disclosure provides a two-dimensional code image segmentation method, comprising: performing interference processing on an original two-dimensional code image through an image enhancement method to construct a two-dimensional code image set having a plurality of interference factors, the two-dimensional code image set comprising a plurality of two-dimensional code noise images, and each two-dimensional code noise image having at least one interference factor; the two-dimensional code image set is utilized to optimize a two-dimensional code segmentation model so as to obtain a target segmentation model used for segmenting a two-dimensional code area under the influence of interference factors, and the target segmentation model utilizes an image segmentation loss function to carry out deviation calculation between the two-dimensional code prediction image and the two-dimensional code expectation image; and according to image features extracted from the target scene image by the target segmentation model, segmenting a background region and a two-dimensional code region in the target scene image to obtain a two-dimensional code image corresponding to the two-dimensional code region. The invention further provides a two-dimensional code image segmentation device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of computer science and technology, and in particular to a method, device, electronic device and storage medium for segmenting a two-dimensional code image. Background Art

[0002] QR code is a widely used coding method in commercial and industrial fields, which can be used to store a large amount of information, such as product information, website, contact information, etc. Separating the QR code from the image makes it easier to identify and decode the two-dimensional code, thereby realizing automated information processing and management.

[0003] However, in actual applications, such as mobile payment, access control management, logistics tracking and other fields, people often scan codes in an unordered manner. At this time, the scanning device is far away from the QR code or at a certain angle to the QR code, so that the acquired QR code image contains a lot of noise and interference information, which is more likely to cause recognition errors or failures. The existing method of recognizing QR codes is not suitable for complex scenes such as light interference, shadow occlusion or blur of QR codes, which reduces the recognition efficiency of QR codes and the user scanning experience in complex scenes. Summary of the invention

[0004] In order to solve at least one of the above technical problems, the present disclosure provides a method, device, electronic device and storage medium for segmenting a two-dimensional code image.

[0005] According to one aspect of the present disclosure, a method for segmenting a two-dimensional code image is proposed, comprising: performing interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, wherein the two-dimensional code image set includes multiple two-dimensional code noise images, and each of the two-dimensional code noise images has at least one of the interference factors; optimizing a two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting a two-dimensional code area under the influence of the interference factors, wherein the target segmentation model calculates the deviation between a two-dimensional code prediction image and a two-dimensional code expected image using an image segmentation loss function; and segmenting a background area and a two-dimensional code area in the target scene image according to image features extracted in the target scene image by the target segmentation model to obtain a two-dimensional code image corresponding to the two-dimensional code area.

[0006] In some embodiments, the original two-dimensional code image is expanded by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, including: performing multiple interference processing on the original two-dimensional code image to form multiple two-dimensional code noise images with at least one interference factor, wherein the interference processing package at least includes rotating, folding and Gaussian blurring the original two-dimensional code image; and integrating each of the two-dimensional code noise images and the original two-dimensional code image to construct a two-dimensional code image set with multiple interference factors.

[0007] In some embodiments, before integrating the respective two-dimensional code noise images and the original two-dimensional code images to construct a two-dimensional code image set having multiple interference factors, it also includes: screening out preliminary screening images having desired interference factors from the plurality of the two-dimensional code noise images and the plurality of the original two-dimensional code images, and constructing the two-dimensional code image set consisting of the plurality of the preliminary screening images.

[0008] In some embodiments, the two-dimensional code segmentation model is optimized using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code area under the influence of the interference factor, including: screening a plurality of complex noise images in the two-dimensional code image set, wherein the complex noise image is an image in the two-dimensional code image set having a target interference factor; and adjusting parameters of the two-dimensional code segmentation model using the complex noise image until the loss value between the output two-dimensional code predicted image and the two-dimensional code expected image is less than a loss threshold, thereby obtaining the target segmented image.

[0009] In some embodiments, the use of the complex noise image to adjust the parameters of the two-dimensional code segmentation model until the loss value between the output two-dimensional code prediction image and the two-dimensional code expected image is less than a loss threshold to obtain the target segmented image, including: performing multi-layer feature extraction on the complex noise image to obtain image features of the complex noise image; segmenting the background prediction area and the two-dimensional code prediction area of ​​the complex noise image according to the image features of the complex noise image to obtain the two-dimensional code prediction area as the two-dimensional code prediction image; using an image segmentation loss function to calculate the loss of the two-dimensional code prediction image and the two-dimensional code expected image to obtain a loss value; and adjusting the parameters of the two-dimensional code segmentation model according to the loss value until the loss value is less than a loss threshold to obtain the target segmented image.

[0010] In some implementations, the image segmentation loss function is expressed as:

[0011]

[0012] Among them, L ohemcerepresents the loss value between the two-dimensional code prediction image and the two-dimensional code expected image, N represents the total number of pixels in the complex noise image, and w i represents the weight of the i-th pixel, p i Represents the predicted probability of the i-th pixel, y i represents the true label of the i-th pixel, and i represents the sequence number of the pixel.

[0013] In some embodiments, the weight w of the i-th pixel is i In y i When it is 1, it takes the preset weight constant α; the weight w of the i-th pixel i In y i When it is 0, the value is 1-α.

[0014] According to another aspect of the present disclosure, a two-dimensional code image segmentation device is proposed, including: an image set expansion module, used to perform interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, wherein the two-dimensional code image set includes multiple two-dimensional code noise images, and each of the two-dimensional code noise images has at least one of the interference factors; a model optimization module, used to optimize the two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code area under the influence of the interference factors, wherein the target segmentation model uses an image segmentation loss function to calculate the deviation between the two-dimensional code prediction image and the two-dimensional code expected image; and an image segmentation module, used to segment the background area and the two-dimensional code area in the target scene image according to the image features extracted by the target segmentation model in the target scene image to obtain the two-dimensional code image corresponding to the two-dimensional code area.

[0015] According to another aspect of the present disclosure, an electronic device is proposed, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the two-dimensional code image segmentation method described in any of the above embodiments.

[0016] According to another aspect of the present disclosure, a readable storage medium is provided, wherein the readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, the method for segmenting a two-dimensional code image described in any of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0018] Figure 1 The figure is a flow chart of a method for segmenting a two-dimensional code image according to an exemplary embodiment of the present disclosure.

[0019] Figure 2 This is a graph of the training set loss when ResNet50 is used as the feature extraction network in an exemplary embodiment of the present disclosure.

[0020] Figure 3 Schematic diagram of the loss curve of the target segmentation model according to an exemplary embodiment of the present disclosure.

[0021] Figure 4 It is the indicator graph of the original U-Net segmentation result.

[0022] Figure 5 It is a segmentation result index graph of the target segmentation model according to an exemplary embodiment of the present disclosure.

[0023] Figure 6 It is a block diagram of a device for segmenting a two-dimensional code image according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0025] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0027] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0028] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0029] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0030] Figure 1 FIG. 1 is a flow chart of a method for segmenting a two-dimensional code image according to an exemplary embodiment of the present disclosure. Figure 1 The two-dimensional code image segmentation method S100 of the present disclosure is described.

[0031] The two-dimensional code image segmentation method S100 proposed in the present disclosure includes:

[0032] Step S102: performing interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors.

[0033] Step S104, optimizing the two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code region under the influence of interference factors, wherein the target segmentation model uses an image segmentation loss function to calculate the deviation between the two-dimensional code prediction image and the two-dimensional code expected image. And

[0034] Step S106, segmenting the background area and the QR code area in the target scene image according to the image features extracted by the target segmentation model in the target scene image, so as to obtain a QR code image corresponding to the QR code area.

[0035] The original QR code image refers to an image containing a QR code extracted from a real scene, which may contain certain interference factors. If the original QR code image captured in a real scene is used as a training sample, it is difficult to obtain a large number of samples. In addition, the types of interference factors in the original QR code image are not rich enough to have a good training effect on the model. The model obtained by training will only have a good prediction effect on the training samples, but will not have a good and stable segmentation result for the QR code in the actual scene.

[0036] The two-dimensional code image set contains multiple two-dimensional code noise images that have been subjected to interference processing, and each two-dimensional code noise image has at least one interference factor, so the two-dimensional code image set contains multiple interference factors. In this way, the target segmentation model obtained using the two-dimensional code image set can adapt to the interference factors in most real scenes, and has a good two-dimensional code segmentation effect in actual scenes. By interference processing the original two-dimensional code image, a large number of two-dimensional code noise images with interference factors can be expanded, so that the two-dimensional code image set has a large number of samples, which can achieve a better model training effect.

[0037] The target segmentation model is a neural network model, which selects the image segmentation loss function (Online hard example mining cross-entropy loss, OhemCELoss) as the loss function. OhemCELoss is a loss function that adds some hyperparameters to the cross entropy loss function CELoss, so that back propagation is performed when the loss function is greater than the given hyperparameter. At the same time, this hyperparameter will also be calculated using probability, and the probability less than the hyperparameter will also be involved in the calculation of the final loss. OhemCELoss realizes online hard sample mining, which not only organically combines the stochastic gradient descent method SGD with the alternating training method, but also has a simple implementation method compared to other loss functions, which can improve the efficiency of network training and detection performance.

[0038] In some embodiments, the specific implementation method of step S102 includes: performing multiple interference processes on the original two-dimensional code image to form multiple two-dimensional code noise images with at least one interference factor, wherein the interference processing package at least includes rotating, folding and Gaussian blurring the original two-dimensional code image; and integrating each two-dimensional code noise image and the original two-dimensional code image to construct a two-dimensional code image set with multiple interference factors.

[0039] In some embodiments, before integrating the various QR code noise images and the original QR code images to construct a QR code image set with multiple interference factors, it also includes: screening out preliminary screening images with desired interference factors from multiple QR code noise images and multiple original QR code images, and constructing a QR code image set consisting of multiple preliminary screening images.

[0040] Considering that there may be some images without interference factors or with weak interference factors in the original QR code image, or there may be some images with weak interference factors after the original QR code image is interfered, by clearing these images, the images in the sample set have complex interference factors, thus ensuring the effectiveness of model training.

[0041] In some embodiments, the specific implementation of step S104 includes: screening multiple complex noise images in the QR code image set, wherein the complex noise image is an image in the QR code image set with target interference factors; and using the complex noise image to adjust the parameters of the QR code segmentation model until the loss value between the output QR code prediction image and the QR code expected image is less than the loss threshold, thereby obtaining the target segmented image.

[0042] The target interference factor refers to a complex factor. A complex noise image with the target interference factor is an image in which it is difficult to extract the QR code area from the QR code image set, such as a blurred and difficult to identify QR code image. This step is mainly to train the model with difficult samples to improve the model's QR code segmentation effect.

[0043] Specifically, the process of using a complex noise image to train a target segmentation image is as follows: performing multi-layer feature extraction on the complex noise image to obtain image features of the complex noise image; segmenting the background prediction area and the two-dimensional code prediction area of ​​the complex noise image according to the image features of the complex noise image to obtain the two-dimensional code prediction area as a two-dimensional code prediction image; using an image segmentation loss function to calculate the loss of the two-dimensional code prediction image and the two-dimensional code expected image, and obtaining a loss value; and adjusting parameters of the two-dimensional code segmentation model according to the loss value until the loss value is less than a loss threshold to obtain the target segmentation image.

[0044] In some embodiments, the image segmentation loss function is expressed as:

[0045]

[0046] Among them, L ohemce represents the loss value between the QR code prediction image and the QR code expected image, N represents the total number of pixels in the complex noise image, and w i represents the weight of the i-th pixel, p i Represents the predicted probability of the i-th pixel, y i represents the true label of the i-th pixel, and i represents the sequence number of the pixel.

[0047] Among them, the weight w of the i-th pixel i In y i When it is 1, it takes the preset weight constant α; the weight w of the i-th pixel i In y i When it is 0, the value is 1-α. The value range of the preset weight constant α is usually between 0.25 and 0.75.

[0048] Combine the following Figures 2 to 5 The overall architecture and effect of the two-dimensional code image segmentation method S100 are introduced in detail.

[0049] This method is based on the segmentation of the QR code image implemented by the U-Net network model, so that the recognition accuracy of the QR code under complex conditions can be close to that under normal conditions, thereby improving the recognition accuracy of the QR code.

[0050] Description of the data collection stage. The data sets used in the present disclosure all use the two-dimensional code images collected by taking photos in real life as the original two-dimensional code images. For example, there are 1020 original two-dimensional code images collected, and the two-dimensional codes in each original two-dimensional code image are not unique. Because the collected original two-dimensional code images are subject to interference factors in nature, including but not limited to: stains, dust, wrinkles, blur, etc., these interference factors will reduce the accuracy of segmentation. In order to solve the overfitting problem in the experimental process, the present disclosure uses a data enhancement method to expand the original data set. The data enhancement method includes rotation, folding, Gaussian blurring, etc., so that the two-dimensional code noise images in the existing data set (i.e., the two-dimensional code image set) are expanded, for example, from 1020 to 8750.

[0051] Description of data preprocessing. The present invention uses programming software (such as labelme, a data annotation software under the programming software anaconda) to annotate each QR code noise image in the collected QR code image set with the collected pictures containing multiple interferences (i.e., the original QR code image), obtains a json file in the lightweight data exchange format json, and then uses the labelme-json-to-datasetjson (converts the mark to a dataset) method built into the programming software to convert the json file into a file in the voc format, obtains a label file in the image file storage format png, and finally stores it in the format of a standard dataset, such as the VOC2007 format.

[0052] Description of the model building stage. This paper adopts the U-Net network model, in which a four-layer downsampling feature extraction network is used, corresponding to four maximum pooling layers. At the same time, the enhanced feature extraction network with jump connections has four layers of anti-pooling layers for upsampling, and finally further feature fusion is performed through jump connections. After the last step of feature fusion is completed, all features are sent to the prediction network. It can be seen

[0053] Figure 2 This is a training set loss curve diagram when ResNet50 is used as the feature extraction network in an exemplary embodiment of the present disclosure.

[0054] like Figure 2 As shown in the figure, the dataset used is 1020 images from the original QR code image set without data enhancement. The model loss function uses the cross entropy loss function. The training effect of the original U_Net network model is 0.012 for the training set and 0.038 for the test set. Obviously, the test set loss is much larger than the training set, and the test set performance is not good, and overfitting occurs. Among them, train loss represents the training set loss value, val loss represents the validation set loss value, smoothtrain loss represents the smoothed training set loss value, and smoothval loss represents the smoothed validation set loss value.

[0055] Based on the above, the prediction results of the original U-Net network model have obvious overfitting problems. In order to reduce the above overfitting phenomenon, the present disclosure optimizes the model in the following two aspects: data enhancement and replacement of loss function. The improvements in the two aspects are described one by one below.

[0056] Regarding data enhancement, considering that the more training sets there are, the lower the probability of overfitting, the present disclosure adopts data enhancement to expand the data set. The data enhancement methods can at least include: rotation, folding, scaling, shearing, Gaussian blur, etc. In subsequent experiments, the data was expanded from the original 1020 original two-dimensional code images to 8750 two-dimensional code noise images. The training set and the test set were divided into 9:1, with 7875 training sets and 875 test sets. After forming the two-dimensional code image set, the experiment was conducted again.

[0057] For the replacement loss function, the loss function used by the original U-Net network is the more common cross entropy loss function, but the image segmentation effect is not ideal. In order to better converge the model, the QR code segmentation model in the present disclosure uses OhemCELoss (image segmentation loss function) as the loss function. This loss function is based on the cross entropy loss function, that is, CELoss, and adds some hyperparameters, so that when the loss function is greater than the established hyperparameter, back propagation is performed. At the same time, this hyperparameter will also be calculated using probability. When the value of the loss function is less than the probability of the hyperparameter, it will also participate in the calculation of the final loss. The hyperparameter is a preset upper limit of the loss function. When the loss function value is less than the hyperparameter, the result of the model is considered to be reliable; otherwise, it is fed back to the model to provide an optimization direction for the adjustment of the parameter weights in the model. The difference between this loss function and the cross entropy loss function is that it realizes online difficult sample mining. Ohem not only organically combines the stochastic gradient descent (SGD) optimization method with alternating training, but also has a relatively simple implementation compared to other loss functions. It can improve the efficiency of network training and detection performance, and is widely used in difficult sample mining scenarios. It is completely suitable for people's daily use of mobile phones, tablets and other devices to scan QR codes, and the data set has been expanded by more than ten times compared to the original experiment through data enhancement. The characteristic of Ohem is that the larger the data set and the higher the difficulty, the more obvious the improvement of Ohem in accuracy will be.

[0058] The image segmentation loss function OhemCELoss is expressed as:

[0059]

[0060] Among them, L ohemce represents the loss value between the QR code prediction image and the QR code expected image, N represents the total number of pixels in the complex noise image, and w i represents the weight of the i-th pixel, p i Represents the predicted probability of the i-th pixel, y i represents the true label of the i-th pixel, and i represents the sequence number of the pixel.

[0061] Among them, the weight w of the i-th pixel i The calculation method is:

[0062]

[0063] That is to say, w i In y i When it is 1, it takes the preset weight constant α; the weight w of the i-th pixel i In y i When it is 0, the value is 1-α. The value range of the preset weight constant α is usually between 0.25 and 0.75.

[0064] Furthermore, the present disclosure trains the model by using a set of QR code images after data enhancement and observes its loss curve.

[0065] Figure 3 Schematic diagram of the loss curve of the target segmentation model according to an exemplary embodiment of the present disclosure.

[0066] like Figure 3 As shown, the horizontal axis is the number of iterations Epoch, and the vertical axis is the loss value Loss. Figure 2 The target segmentation model with the image segmentation loss function OhemCELoss as the loss function has a better convergence trend, and the test set loss is also reduced from 0.04 to about 0.01.

[0067] Figure 4 It is the indicator graph of the original U-Net segmentation result.

[0068] If the original U-Net network model without data enhancement and loss function replacement is used to segment the QR code image, the MIoU and mPA indexes are 91.11% and 95.15% respectively. Figure 4 The figure shows the similarity between the model result and the real annotation between the QR code image erweima and the background image background, as well as the pixel accuracy between the QR code image and the background image, where pixel accuracy is the ratio of the number of correctly classified pixels to the total number of pixels. Since the main research of this disclosure is the segmentation of QR codes, only the QR codes are circled when annotating the data, and the rest are set as background, so there are only two categories in the generated bar chart.

[0069] Figure 5 It is a segmentation result index graph of the target segmentation model according to an exemplary embodiment of the present disclosure.

[0070] In this disclosure, the QR code image set after data enhancement is used as the sample set, and OhemCELoss is used as the loss function of the target segmentation image to segment the QR code noise image. The MIoU and MPA index scores obtained by the experimental results are 99.17% and 99.57% respectively. Figure 5 shown.

[0071] Obviously, the present invention uses OhemCELoss as the loss function and performs data enhancement at the same time. The segmentation results have significant improvements in both MIoU and MPA evaluation indicators, which further illustrates that the method of the present invention is more accurate than the QR code image segmented by the original U-Net network.

[0072] The results of the segmentation experiment show that the segmentation accuracy of the trained target segmentation model is more accurate when the data-enhanced QR code image set is used as the training set and OhemCELoss is used as the target segmentation model loss function. The edge contour segmentation of the QR code image is clearer, especially when multiple QR codes appear in the image. This method can recognize and segment them, and can also segment the parts that appear in the image where the QR code is not fully displayed. The experimental results show that this method significantly improves the accuracy of QR code image segmentation.

[0073] The segmentation method of the two-dimensional code image proposed in the present disclosure generates multiple two-dimensional code noise images by interfering with the original two-dimensional code image, thereby increasing the number of training samples, and training the model by a set of two-dimensional code images with multiple interference factors, thereby avoiding the problem of overfitting and ensuring the universality of the target segmentation model. By setting OhemCELoss as the loss function of the model, the ability of the model to remove interference factors is improved, thereby improving the accuracy.

[0074] Figure 6 It is a block diagram of a device for segmenting a two-dimensional code image according to an exemplary embodiment of the present disclosure.

[0075] like Figure 6As shown, the present disclosure proposes a two-dimensional code image segmentation device 1000, including: an image set expansion module 1002, used to perform interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, wherein the two-dimensional code image set includes multiple two-dimensional code noise images, and each two-dimensional code noise image has at least one interference factor; a model optimization module 1004, used to optimize the two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code area under the influence of the interference factors, wherein the target segmentation model uses an image segmentation loss function to calculate the deviation between the two-dimensional code prediction image and the two-dimensional code expected image; and an image segmentation module 1006, used to segment the background area and the two-dimensional code area in the target scene image according to the image features extracted by the target segmentation model in the target scene image to obtain the two-dimensional code image corresponding to the two-dimensional code area.

[0076] The various modules of the two-dimensional code image segmentation device 1000 are arranged to implement the various steps of the two-dimensional code image segmentation method. The principles and execution processes thereof can be referred to above and will not be described in detail here.

[0077] The device may include a corresponding module for performing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be performed by a corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specifically configured to perform the corresponding steps, or implemented by the processor 1300 configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by the processor 1300, or implemented by some combination.

[0078] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnected buses 1100 and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various other circuits 1400 including one or more processors 1300, memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0079] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus 1100 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus 1100.

[0080] Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, in which functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong. The processor 1300 performs the various methods and processes described above. For example, the method implementation in the present disclosure may be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory 1300. In some embodiments, part or all of the software program may be loaded and / or installed via the memory 1300 and / or a communication interface. When the software program is loaded into the memory 1300 and executed by the processor 1300, one or more steps in the method described above may be performed. Alternatively, in other embodiments, the processor 1300 may be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0081] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including processor 1300, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0082] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0083] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0084] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0085] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0086] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for segmenting a two-dimensional code image, characterized in that: include: Performing interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, wherein the two-dimensional code image set includes multiple two-dimensional code noise images, and each of the two-dimensional code noise images has at least one of the interference factors; Optimizing a two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting a two-dimensional code region under the influence of the interference factor, wherein the target segmentation model calculates a deviation between a two-dimensional code prediction image and a two-dimensional code expected image using an image segmentation loss function; and According to the image features extracted by the target segmentation model in the target scene image, the background area and the two-dimensional code area in the target scene image are segmented to obtain a two-dimensional code image corresponding to the two-dimensional code area.

2. The method for segmenting a two-dimensional code image according to claim 1, characterized in that: The interference processing of the original two-dimensional code image by the image enhancement method to construct a two-dimensional code image set with multiple interference factors includes: Performing multiple interference processing on the original two-dimensional code image to form multiple two-dimensional code noise images having at least one interference factor, wherein the interference processing comprises at least rotating, folding and Gaussian blurring the original two-dimensional code image; and The two-dimensional code noise images and the original two-dimensional code images are integrated to construct a two-dimensional code image set with multiple interference factors.

3. The method for segmenting a two-dimensional code image according to claim 2, characterized in that: Before integrating the two-dimensional code noise images and the original two-dimensional code images to construct a two-dimensional code image set with multiple interference factors, the method further includes: A preliminary screening image with expected interference factors is screened out from the plurality of the two-dimensional code noise images and the plurality of the original two-dimensional code images, and the two-dimensional code image set consisting of the plurality of the preliminary screening images is constructed.

4. The method for segmenting a two-dimensional code image according to claim 1, characterized in that: The optimizing the two-dimensional code segmentation model by using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code region under the influence of the interference factor includes: Screening a plurality of complex noise images in the two-dimensional code image set, wherein the complex noise images are images in the two-dimensional code image set having target interference factors; and The complex noise image is used to adjust the parameters of the two-dimensional code segmentation model until the loss value between the output two-dimensional code prediction image and the two-dimensional code expected image is less than a loss threshold, thereby obtaining the target segmentation image.

5. The method for segmenting a two-dimensional code image according to claim 4, characterized in that: The method of adjusting the parameters of the two-dimensional code segmentation model by using the complex noise image until the loss value between the output two-dimensional code prediction image and the two-dimensional code expected image is less than a loss threshold to obtain the target segmentation image includes: Performing multi-layer feature extraction on the complex noise image to obtain image features of the complex noise image; According to the image features of the complex noise image, segmenting the background prediction area and the two-dimensional code prediction area of ​​the complex noise image to obtain the two-dimensional code prediction area as the two-dimensional code prediction image; Using an image segmentation loss function, performing loss calculation on the predicted two-dimensional code image and the expected two-dimensional code image, and obtaining a loss value; and According to the loss value, the parameters of the two-dimensional code segmentation model are adjusted until the loss value is less than a loss threshold, thereby obtaining the target segmentation image.

6. The method for segmenting a two-dimensional code image according to claim 5, characterized in that: The image segmentation loss function is expressed as: Among them, L ohemce represents the loss value between the two-dimensional code prediction image and the two-dimensional code expected image, N represents the total number of pixels in the complex noise image, and w i represents the weight of the i-th pixel, p i Represents the predicted probability of the i-th pixel, y i represents the true label of the i-th pixel, and i represents the sequence number of the pixel.

7. The method for segmenting a two-dimensional code image according to claim 6, characterized in that: The weight w of the i-th pixel i In y i When it is 1, it takes the preset weight constant α; the weight w of the i-th pixel i In y i When it is 0, the value is 1-α.

8. A device for segmenting a two-dimensional code image, characterized in that: include: An image set expansion module, used for performing interference processing on the original two-dimensional code image by an image enhancement method to construct a two-dimensional code image set with multiple interference factors, wherein the two-dimensional code image set includes multiple two-dimensional code noise images, each of which has at least one of the interference factors; a model optimization module, configured to optimize the two-dimensional code segmentation model using the two-dimensional code image set to obtain a target segmentation model for segmenting the two-dimensional code region under the influence of the interference factor, wherein the target segmentation model calculates the deviation between the two-dimensional code prediction image and the two-dimensional code expected image using an image segmentation loss function; and The image segmentation module is used to segment the background area and the QR code area in the target scene image according to the image features extracted by the target segmentation model in the target scene image, so as to obtain the QR code image corresponding to the QR code area.

9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory, so that the processor executes the two-dimensional code image segmentation method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the two-dimensional code image segmentation method according to any one of claims 1 to 7.

Images