Method and device for adjusting network structure, storage medium and electronic apparatus

Abstract

The application discloses a network structure adjusting method and device under an artificial intelligence scene, a storage medium and an electronic device, and relates to computer vision technology, mechanical learning and other technologies. The method comprises the following steps: in the case that a target network structure for restoring a character image is acquired, compressing N-layer convolution structures included in the target network structure according to M compression modes to obtain P candidate network structures; performing character restoration on a first image in each group of sample character images through the P candidate network structures respectively to obtain a restored character image corresponding to the first image; comparing each restored character information with corresponding restored sample character information in sequence to obtain a character similarity corresponding to the restored character information; and replacing the N-layer convolution structures in the target network structure with a candidate network structure whose character similarity reaches a similarity threshold. The application solves the technical problem of poor network structure adjustment accuracy.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a network structure adjustment method, apparatus, storage medium, and electronic device. Background Technology

[0002] In some practical applications, the real-time performance of the algorithm model is required to be high. Therefore, it is necessary to compress and accelerate the model while maintaining its accuracy as much as possible. For example, when compressing the network using the neural architecture search method, a certain number of channels are randomly selected and activated in the layers other than the input and output layers to obtain a series of sub-models.

[0003] Among the aforementioned series of sub-models, the best-performing sub-model is determined. In existing technologies, the Fréchet distance is often used as an indicator to evaluate the performance of a sub-model. The Fréchet distance is calculated by extracting features from the generated image and the original image using the algorithm model, and then calculating the Fréchet distance between the features as an indicator. The smaller the distance, the better the quality of the generated image.

[0004] However, in the field of image processing, the performance of a sub-model is not determined by visual quality. Consequently, the aforementioned method of using FID distance as the evaluation metric for sub-models cannot guarantee the selection of the best-performing sub-model suitable for image processing from a series of sub-models.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a network structure adjustment method, apparatus, storage medium, and electronic device to at least solve the technical problem of poor accuracy in network structure adjustment.

[0007] According to one aspect of the present invention, a network structure adjustment method is provided, comprising: upon obtaining a target network structure for restoring a character image, compressing the N convolutional layers included in the target network structure according to M compression methods to obtain P candidate network structures, wherein the convolutional layers included in the candidate network structures are N layers, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1; and obtaining K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying the sample character to be restored. The first image and the second image carrying the restored sample characters are given, where K is a positive integer greater than or equal to 1. The first image in each group of sample character images is restored using the above P candidate network structures to obtain the restored character image corresponding to the first image. When the restored character information is identified from the restored character image, each restored character information is compared with the character information of the corresponding restored sample character to obtain the character similarity corresponding to the restored character information. The N-layer convolutional structure in the target network structure is replaced with the candidate network structure whose character similarity reaches the similarity threshold.

[0008] According to another aspect of the present invention, a network structure adjustment apparatus is also provided, comprising: a compression unit, configured to, upon obtaining a target network structure for restoring a character image, compress N layers of convolutional structures included in the target network structure according to M compression methods to obtain P candidate network structures, wherein the convolutional structures included in the candidate network structures are N layers, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1; and a first acquisition unit, configured to acquire K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying a sample character to be restored and a... A second image carrying restored sample characters, where K is a positive integer greater than or equal to 1; a restoration unit, used to restore the characters in the first image of each group of sample character images through the above P candidate network structures, to obtain the restored character image corresponding to the first image; a comparison unit, used to compare each restored character information with the character information of the corresponding restored sample character in turn when the restored character information is identified from the restored character image, to obtain the character similarity corresponding to the restored character information; and an adjustment unit, used to replace the N-layer convolutional structure in the target network structure with the candidate network structure whose character similarity reaches the similarity threshold.

[0009] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described network structure adjustment method at runtime.

[0010] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the network structure adjustment method described above through the computer program.

[0011] In this embodiment of the invention, when a target network structure for restoring character images is obtained, the N convolutional layers included in the target network structure are compressed using M compression methods to obtain P candidate network structures. Each candidate network structure includes N convolutional layers, where N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1. K sets of sample character images are obtained, where each set includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, where K is a positive integer greater than or equal to 1. The first image in each set of sample character images is restored using the P candidate network structures to obtain the restored character image corresponding to the first image. When the restored character information is identified from the restored character image, each restored character information is sequentially compared with the character information of the corresponding restored sample character to obtain the character similarity corresponding to the restored character information. The N-layer convolutional structure in the target network structure is replaced with the candidate network structure whose character similarity reaches the similarity threshold. In the evaluation process of the compressed convolutional structure in the field of character restoration, character recognition information is used as the evaluation index to determine the optimal compressed convolutional structure, so as to adjust the overall network structure used for character restoration. This achieves the technical goal that the adjusted network structure can meet the compression requirements while ensuring high restoration accuracy, thereby improving the restoration accuracy of the compressed network structure and solving the technical problem of poor accuracy in network structure adjustment. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0013] Figure 1 This is a schematic diagram of an application environment for an optional network structure adjustment method according to an embodiment of the present invention;

[0014] Figure 2This is a schematic diagram of a flowchart of an optional network structure adjustment method according to an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of an optional network structure adjustment method according to an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of another optional network structure adjustment method according to an embodiment of the present invention;

[0017] Figure 5 This is a schematic diagram of another optional network structure adjustment method according to an embodiment of the present invention;

[0018] Figure 6 This is a schematic diagram of another optional network structure adjustment method according to an embodiment of the present invention;

[0019] Figure 7 This is a schematic diagram of an optional network structure adjustment device according to an embodiment of the present invention;

[0020] Figure 8 This is a schematic diagram of another optional network structure adjustment device according to an embodiment of the present invention;

[0021] Figure 9 This is a schematic diagram of another optional network structure adjustment device according to an embodiment of the present invention;

[0022] Figure 10 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0026] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0027] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0028] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0029] The solutions provided in this application involve artificial intelligence technologies such as computer vision and machine learning, and are specifically illustrated through the following embodiments:

[0030] According to one aspect of the present invention, a network structure adjustment method is provided. Optionally, as an optional implementation, the above-described network structure adjustment method may be applied to, but is not limited to, [examples of network structure adjustment methods]. Figure 1 The environment shown may include, but is not limited to, user equipment 102, network 110, and server 112. User equipment 102 may include, but is not limited to, a display 108, a processor 106, and a memory 104. Optionally, the display 108 may, but is not limited to, display a restoration input interface and a restoration output interface, wherein the character image to be restored is acquired on the restoration input interface, and the restored character image is displayed on the restoration output interface.

[0031] The specific process can be summarized in the following steps:

[0032] Step S102: User equipment 102 acquires the character image to be restored;

[0033] In steps S104-S106, user equipment 102 sends the character image to be restored to server 112 via network 110;

[0034] In step S108, the server 112 processes the character image to be restored through the processing engine 116 to generate the target restoration result.

[0035] In steps S110-S112, server 112 sends the target restoration result to user equipment 102 via network 110. Processor 106 in user equipment 102 processes the target restoration result into a displayable character image and displays it on display 108, and stores the target restoration result in memory 104.

[0036] remove Figure 1Beyond the illustrated example, the above steps can be performed independently by user equipment 102. That is, user equipment 102 can perform steps such as processing the character image to be restored and generating the target restoration result, thereby reducing the processing load on the server. User equipment 102 includes, but is not limited to, handheld devices (such as mobile phones), laptops, desktop computers, etc. This invention does not limit the specific implementation of user equipment 102.

[0037] Alternatively, as an optional implementation, such as Figure 2 As shown, the network structure adjustment methods include:

[0038] S202, after obtaining the target network structure for restoring the character image, compress the N convolutional layers included in the target network structure according to M compression methods to obtain P candidate network structures, wherein the convolutional layers included in the candidate network structure are N layers, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1.

[0039] S204, obtain K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, and K is a positive integer greater than or equal to 1;

[0040] S206, Using P candidate network structures, character restoration is performed on the first image in each group of sample character images to obtain the restored character image corresponding to the first image;

[0041] S208, when the restored character information is identified from the restored character image, each restored character information is compared with the character information of the corresponding restored sample character in turn to obtain the character similarity corresponding to the restored character information;

[0042] S210, replace the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold.

[0043] Optionally, in this embodiment, the network structure adjustment method can be applied, but is not limited to, in the scenario of text restoration. It determines the candidate network structure with the best performance among candidate network structures with various convolutional structure combinations obtained by various compression methods, and adjusts the N-layer convolutional structure in the target network structure based on the candidate network structure with the best performance. This ensures that the adjusted target network structure meets the compression requirements while guaranteeing a certain level of restoration accuracy. In other words, by using the above network structure adjustment method, the running speed of the target network result can be accelerated without significantly reducing the restoration accuracy of the target network structure, thereby reducing the demand on GPU memory and improving the speed and throughput in practical applications.

[0044] Optionally, in this embodiment, the target network structure may be, but is not limited to, a convolutional neural network structure for restoring character images. The convolutional neural network structure may be, but is not limited to, a type of feedforward neural network that includes convolutional computation and has a deep structure, such as WaveNet network structure, UNet network structure, LeNet-4 network structure, ZFNet network structure, VGGNet network structure, residual neural network structure, etc.

[0045] Optionally, in this embodiment, the target network structure may include, but is not limited to, at least one input structure, at least one output structure, and at least two intermediate structures, wherein the intermediate structures include at least N convolutional structures. Optionally, taking the UNet network structure as an example, the input to the UNet network structure may be, but is not limited to, a 3-channel color smeared image, and the output of the UNet network structure may be, but is not limited to, a 3-channel restored color image.

[0046] Optionally, in this embodiment, since the real-time requirements for the reconstruction of the target network structure are high in some practical applications, it is necessary to compress and accelerate the N-layer convolutional deconstruction of the target network structure while maintaining the reconstruction accuracy as much as possible. For example, Neural Architecture Search (NAS) is used to perform the above compression and acceleration, and a certain number of channels are randomly selected to activate the other layers (e.g., convolutional layers) except for the input and output layers to obtain a series of convolutional layer combinations.

[0047] Furthermore, to limit the number of convolutional layer combinations, the number of channels searchable for each convolutional structure is generally limited. For example, for a convolutional structure with a total of 64 channels, the first 32, 48, or 64 channels can be activated (e.g., randomly). In other words, after performing the above channel number selection on the channel number of each convolutional structure in an N-layer convolutional structure, an N-layer convolutional structure with a certain combination of channel numbers is obtained. Optionally, the compression method can be, but is not limited to, performing the same round of channel number selection on the channel number of each convolutional structure to obtain an N-layer convolutional structure with a new combination of channel numbers, where different compression methods correspond to different combinations of channel numbers.

[0048] Optionally, in this embodiment, assuming there are O types of channel number selection and N layers of convolutional structure, after performing M compression methods on the N-layer convolutional structure, it is possible to obtain, but is not limited to, O to the power of N different combinations of channel number. In addition, taking the UNet network structure as an example, its network structure is a symmetrical U-shaped structure, and the latter half needs to be spliced ​​with the channel features of the former half. Since UNet is symmetrically spliced, if a certain layer in the former half selects a certain number of channels, its symmetrical layer is automatically determined to have twice the number of channels. Therefore, it is possible to obtain, but is not limited to, O to the power of (1 / 2×N) different combinations of channel number.

[0049] To further illustrate, scenarios for obtaining candidate network structures include, for example... Figure 3 As shown, the target network structure 302 includes N convolutional layers, and each convolutional layer can select three types of channel counts (32, 48, 64). Assuming that the channel count of each convolutional layer is selected to determine the current channel count of each layer, for example, the current channel count of the first convolutional layer 3022 is 32, the current channel count of the second convolutional layer 3024 is 48, and the current channel count of the third convolutional layer 3026 is 64, then the N-layer convolutional structure of the current convolutional layer combination is determined, completing the process. The current compression method is executed to obtain the corresponding candidate network structure, which consists of a first convolutional structure 3022 with 32 channels, a second convolutional structure 3024 with 48 channels, and a third convolutional structure 3026 with 64 channels. The current compression method includes selecting 32 channels for the first convolutional structure 3022, selecting 48 channels for the second convolutional structure 3024, and selecting 64 channels for the third convolutional structure 3026.

[0050] Optionally, in this embodiment, each set of sample character images includes a first image and a second image with a corresponding relationship. The second image may be, but is not limited to, the original character image, and the first image may be, but is not limited to, a character image that has been blurred, covered, or smeared based on the original character image. The first image may be, but is not limited to, a clear character that can be identified, and the second image may be, but is not limited to, a clear character that cannot be identified, or a character information that cannot be identified that does not reach the clarity threshold.

[0051] Optionally, in this embodiment, the restored character information can be used, but is not limited to, to represent the character recognition information obtained by character recognition. Character recognition can be, but is not limited to, using optical character recognition (OCR) technology. OCR text recognition can be, but is not limited to, the process by which an electronic device examines a character image and then uses a character recognition method to translate the shape into computer text. That is, the process of scanning text data and then analyzing and processing the image file to obtain text and layout information.

[0052] It should be noted that in the field of text restoration, the restoration effect of the target network model is evaluated using the OCR text recognition results as the evaluation index. The candidate network structure with the best performance is determined, and the target network structure is adjusted to achieve the best network structure that balances real-time restoration and restoration accuracy.

[0053] Optionally, in this embodiment, character similarity can be represented by text edit distance, which can be, but is not limited to, Levenshtein distance, which refers to the minimum number of editing operations required to transform one string or character into another. The larger the distance between them, the lower the similarity; conversely, the smaller the edit distance, the lower the similarity. Permitted editing operations include replacing one character with another, inserting a character, deleting a character, etc.

[0054] It should be noted that, given the target network structure for restoring character images, the N convolutional layers in the target network structure are compressed using M compression methods to obtain P candidate network structures. Each candidate network structure contains N convolutional layers, where N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1. K sets of sample character images are obtained, each set including a first image carrying the character to be restored and a second image carrying the restored character, where K is a positive integer greater than or equal to 1. The first image in each set of sample character images is restored using the P candidate network structures to obtain the restored character image corresponding to the first image. When the restored character information is identified from the restored character image, each restored character information is compared sequentially with the character information of the corresponding restored sample character to obtain the character similarity corresponding to the restored character information. The N convolutional layers in the target network structure are replaced with candidate network structures whose character similarity reaches a similarity threshold.

[0055] Optionally, in this embodiment, since a network structure involving multiple stages is involved, the network structure of these multiple stages can be trained sequentially, but is not limited to. For example, the target network structure can be pre-trained first, and then the adjusted target network structure can be trained a second time. The training samples for pre-training and secondary training can be, but are not limited to, the same or different sample character images. For example, M+N first images and corresponding second images can be obtained. M images (generally M>20000) are used as training data for pre-training, and the remaining N images (generally M>1000) are used as test data for adjusting the target network structure. The training data for secondary training can be freely combined according to needs. For example, M images can be used as training data for secondary training, or M+N images can all be used as training data for secondary training.

[0056] Furthermore, the network structures for these multiple stages can be trained separately, but not limited to, training the target network structure and the adjusted target network structure, or training only the adjusted target network structure, or training only the target network structure. This is merely an example and not a limitation.

[0057] According to the embodiments provided in this application, when a target network structure for restoring character images is obtained, the N convolutional layers included in the target network structure are compressed using M compression methods to obtain P candidate network structures. Each candidate network structure includes N convolutional layers, where N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1. K sets of sample character images are obtained, where each set of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, where K is a positive integer greater than or equal to 1. The first image in each set of sample character images is restored using the P candidate network structures to obtain a character restoration profile of the first image. The process involves: reconstructing the character image; identifying the reconstructed character information from the reconstructed character image; sequentially comparing each reconstructed character information with the corresponding reconstructed sample character information to obtain the character similarity corresponding to the reconstructed character information; replacing the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold; using character recognition information as an evaluation index in the evaluation process of the compressed convolutional structure in the field of character reconstruction to determine the optimal compressed convolutional structure, thereby adjusting the overall network structure used for character reconstruction, thus achieving the technical objective of ensuring high reconstruction accuracy while meeting the compression requirements of the adjusted network structure, thereby improving the reconstruction accuracy of the compressed network structure.

[0058] As an optional approach, the N convolutional layers in the target network structure are compressed using M compression methods to obtain P candidate network structures, including:

[0059] S1, compress the N-layer convolutional structure according to M compression methods to obtain M candidate network structures, wherein the candidate network structures include N layers of convolutional structures;

[0060] S2, among the M candidate network structures, P candidate network structures whose target computational cost reaches the computational threshold are determined as P candidate network structures, where the target computational cost is used to represent the computational cost of one convolution of the candidate network structure.

[0061] Optionally, in this embodiment, the computational cost of different convolutional structures is different. Generally, convolutional structures with more channels have a higher computational cost and slower speed, while those with fewer channels are faster. The computational cost of each convolutional structure can be calculated using, but is not limited to, MACs (Multiply-Accumulate Operations). Furthermore, in practical applications, a threshold can be determined based on computational requirements. When evaluating and selecting P candidate network structures from M alternative network structures, only those meeting the computational cost or speed requirements are chosen.

[0062] It should be noted that the N-layer convolutional structure is compressed using M compression methods to obtain M candidate network structures, where each candidate network structure includes N layers of convolutional structures. P candidate network structures among the M candidate network structures whose target computational cost reaches a computational threshold are determined as P candidate network structures, where the target computational cost represents the computational cost of one convolution of the candidate network structure. Optionally, the M candidate network structures may include, but are not limited to, the P candidate network structures, where P may be, but is not limited to, less than or equal to M.

[0063] To further illustrate, an optional assumption is to use a blurred image (first image) and the corresponding original image (second image) as training samples (K sets of sample character images) to obtain a scenario for adjusting the candidate network structure of the target network. Figure 4 As shown, the specific steps are as follows:

[0064] Step S402: Input the blurred image and the original image as training samples;

[0065] Step S404: Determine the convolution structure and the corresponding search space;

[0066] Step S406: Train the target network structure and randomly activate convolutional structures to obtain multiple alternative network structures;

[0067] Step S408: Based on the computational load, preliminary screening of candidate network structures that meet the requirements is performed;

[0068] Step S410: Evaluate the quality of each candidate network structure using text edit distance and select the best few candidate network structures;

[0069] Step S412: Fine-tune the candidate network structure to determine the final candidate network structure;

[0070] Step S414: Save the fine-tuned candidate network structure.

[0071] The embodiments provided in this application compress an N-layer convolutional structure using M compression methods to obtain M candidate network structures, where each candidate network structure includes an N-layer convolutional structure. P candidate network structures among the M candidate network structures whose target computational cost reaches a computational threshold are identified as P candidate network structures. The target computational cost represents the computational cost of a single convolution of the candidate network structure. This achieves the goal of selecting candidate network structures based on the computational cost of the convolutional structure, thereby improving the flexibility of obtaining candidate network structures.

[0072] As an alternative approach, the N-layer convolutional structure is compressed using M compression methods to obtain M candidate network structures, including:

[0073] The number of channels in each of the N convolutional layers is compressed using M compression methods to obtain M candidate network structures. Each candidate network structure has a different channel number sequence, which is a sequence synthesized from the channel arrays of each convolutional layer in the candidate network structures. The number of channels is positively correlated with the target computational cost.

[0074] Optionally, in this embodiment, the channel can also be, but is not limited to, a feature map. The convolution result can, but is not limited to, interact between channels, and then generate new channels in the next layer, such as the 1×1 convolution operation used in the Incept-Net network structure. Furthermore, the number of channels can, but is not limited to, depend on the number of filters configured in the convolution result.

[0075] It should be noted that the number of channels in each of the N convolutional layers is compressed using M compression methods to obtain M candidate network structures. The channel number sequence corresponding to each candidate network structure is different. The channel number sequence is a sequence synthesized from the channel array of each convolutional layer in the candidate network structure. The number of channels is positively correlated with the target computational cost.

[0076] To further illustrate, for a convolutional structure with a total of 64 channels, one can choose to activate (e.g., randomly) the first 32, 48, or 64 channels. Assuming the target network structure is a symmetrical U-shaped structure, the adjustment scenarios for the N convolutional layers in the target network structure are as follows: Figure 5 As shown in (a), besides the input and output channels, the number of other channels are 64, 128, 246, 412, 412, 412, 412, 412, 412, 1024, 1024, 1024, 1024, 412, 246, and 128, respectively. Generally, the search space of each convolutional layer can be set to the first 50%, 75%, and 100% of its channels, that is, each convolutional layer can have three channel number options, such as... Figure 5 As shown in (b), a 64-channel layer can randomly activate its first 32, 48, and 64 channels. In addition, since it is a U-shaped structure that is symmetrically spliced, if a certain number of channels is selected in a certain layer in the first half, the symmetrical layer will automatically determine twice the number of channels. Therefore, the maximum number of candidate network structures that can be obtained here is 3^8 = 6561.

[0077] Furthermore, once all candidate network structures are obtained, performance evaluation of each candidate network structure begins. The evaluation metric is the character similarity determined based on the OCR recognition results. A specific implementation may include, but is not limited to, a test set (K sets of sample character images) containing multiple smeared images (first images) and unsmeared original images (second images). Each candidate network structure is used to restore the smeared images, and then the OCR algorithm is used to perform character recognition on both the restored and original images. The edit distance of the recognized characters is calculated and used as the performance evaluation metric for each candidate network structure. A smaller edit distance indicates that the recognition result of the generated restored image is closer to the recognition result of the original image, i.e., a better restoration effect. Based on this metric, several candidate network structures with the smallest edit distance within a certain computational range are selected.

[0078] Furthermore, after obtaining several candidate network structures, it is possible, but not limited to, training each of the candidate network structures separately and using text edit distance to evaluate the performance of the fine-tuned candidate network structures to obtain the final candidate network structure, which can then replace the N-layer convolutional structure in the target network structure.

[0079] The embodiments provided in this application compress the number of channels in each convolutional layer of an N-layer convolutional structure according to M compression methods to obtain M candidate network structures. Each candidate network structure has a different channel number sequence, which is a sequence synthesized from the channel array of each convolutional layer in the candidate network structure. The number of channels is positively correlated with the target computational load, thus achieving the purpose of compressing the network structure according to the number of channels and improving the compression efficiency of the network structure.

[0080] As an optional approach, before compressing the N convolutional layers in the target network structure according to M compression methods, the following steps are included:

[0081] S1, obtain F groups of sample character images, where each group of sample character images in the F groups includes a first image and a second image, and F is a positive integer greater than or equal to 1;

[0082] S2, input the character images of group F samples into the initial target network structure to train and obtain the target network structure.

[0083] Optionally, in this embodiment, the N-layer convolutional structure in the initial target network structure may be compressed in a random compression manner, or may not be compressed at all.

[0084] To further illustrate, alternatively, after compressing the N convolutional layers in the target network structure using M compression methods to obtain M candidate network structures, F sets of sample character images are input into the initial target network structure to train the target network structure; or, alternatively, before compressing the N convolutional layers in the target network structure using the M compression methods, F sets of sample character images are input into the initial target network structure to train the target network structure; or, after compressing the N convolutional layers in the target network structure using the M compression methods to obtain P candidate network structures, F sets of sample character images are input into the initial target network structure to train the target network structure. These are merely examples and are not intended to be limiting.

[0085] It should be noted that F sets of sample character images are obtained, where each set of sample character images in the F sets of sample character images includes a first image and a second image, and F is a positive integer greater than or equal to 1; the F sets of sample character images are input into the initial target network structure to train and obtain the target network structure.

[0086] To illustrate further, one could, for example, randomly compress the N convolutional layers in the initial target network structure and input F sets of sample character images into the randomly compressed initial target network structure to train the target network structure.

[0087] The embodiments provided in this application obtain F sets of sample character images, wherein each set of sample character images in the F sets includes a first image and a second image, and F is a positive integer greater than or equal to 1; the F sets of sample character images are input into the initial target network structure to train the target network structure, thereby achieving the purpose of flexibly training the target network structure and improving the training efficiency of the target network structure.

[0088] As an optional approach, after inputting the F sets of sample character images into the initial target network structure to train and obtain the target network structure, the following steps are included:

[0089] Input K sets of sample character images into the target network structure to train and obtain a trained target network structure.

[0090] Optionally, in this embodiment, the target network structure trained from F sets of sample character images can be, but is not limited to, pre-trained. Based on this, it can also be, but is not limited to, further trained from K sets of sample character images to obtain a trained target network structure. In other words, the training of the target network structure obtained from the F sets of sample character images is insufficient and requires further fine-tuning. This fine-tuning can be, but is not limited to, using K sets of sample character images to perform secondary training on the acquired candidate network structure, using edit distance to evaluate the quality of the fine-tuned candidate network structure, and obtaining the final candidate network structure as the compressed candidate network structure.

[0091] It should be noted that K sets of sample character images are input into the target network structure to train and obtain a trained target network structure.

[0092] Through the embodiments provided in this application, K sets of sample character images are input into the target network structure to train a trained target network structure, thereby achieving the purpose of flexibly training the target network structure and improving the training efficiency of the target network structure.

[0093] As an optional approach, K sets of sample character images are input into the replaced target network structure to train a trained target network structure, including:

[0094] S1, Repeat the following steps until the trained target network structure is obtained:

[0095] S2, determine the current sample character image from K groups of sample character images, and determine the current target network structure, wherein the current sample character image includes the current first image and the current second image;

[0096] S3, obtain the current restoration result through the current target network structure, wherein the current restoration result includes the restored character information corresponding to the current first image;

[0097] S4, if the current restoration result has not reached the convergence condition, obtain the next sample character image as the current sample character image;

[0098] S5. If the current restoration result meets the convergence condition, determine the current target network structure as the trained target network structure.

[0099] Optionally, in this embodiment, text editing distance can be used as the criterion for determining convergence. For example, if the current restoration result indicates that the text editing distance between the restored character information corresponding to the current first image and the restored character information corresponding to the corresponding second image reaches a distance threshold, the convergence condition can be determined to have been met.

[0100] It should be noted that the following steps are repeated until a trained target network structure is obtained: The current sample character image is determined from the K groups of sample character images, and the current target network structure is determined, where the current sample character image includes the current first image and the current second image; the current restoration result is obtained through the current target network structure, where the current restoration result includes the restored character information corresponding to the current first image; if the current restoration result does not meet the convergence condition, the next sample character image is obtained as the current sample character image; if the current restoration result meets the convergence condition, the current target network structure is determined as the trained target network structure.

[0101] The embodiments provided in this application involve repeating the following steps until a trained target network structure is obtained: The current sample character image is determined from K groups of sample character images, and the current target network structure is determined, wherein the current sample character image includes the current first image and the current second image; the current restoration result is obtained through the current target network structure, wherein the current restoration result includes the restored character information corresponding to the current first image; if the current restoration result does not reach the convergence condition, the next sample character image is obtained as the current sample character image; if the current restoration result reaches the convergence condition, the current target network structure is determined as a trained target network structure, thus achieving the purpose of training a complete target network structure and improving the training completeness of the target network structure.

[0102] As an optional approach, the character information of each restored character is compared with the character information of the corresponding restored sample character, including:

[0103] The information difference between each restored character and the corresponding restored sample character is obtained. The information difference is inversely correlated with the character similarity.

[0104] Optionally, in this embodiment, the information difference may include, but is not limited to, text information difference, graphic information difference, feature information difference, stroke information difference, etc. Taking text information difference as an example, assuming that the text information difference is used to represent the difference between two texts (or characters), the text information difference may be represented by, but is not limited to, text edit distance.

[0105] It should be noted that the information difference between each restored character and the corresponding restored sample character is obtained, and the information difference is inversely correlated with the character similarity.

[0106] The embodiments provided in this application obtain the information difference between each restored character information and the corresponding restored sample character information. The information difference is inversely correlated with the character similarity, thereby achieving the purpose of using the information difference between character information as an evaluation index of the convolutional structure and improving the restoration accuracy of the obtained convolutional structure.

[0107] As an optional approach, after replacing the N convolutional layers in the target network structure with candidate network structures whose character similarity reaches a similarity threshold, the following steps are taken:

[0108] S1, Obtain the image of the character to be restored;

[0109] S2, Input the character image to be restored into the adjusted target network structure;

[0110] S3, obtain the target restoration result output by the adjusted target network structure, wherein the target restoration result includes the restored character information corresponding to the character image to be restored.

[0111] Optionally, in this embodiment, the adjusted target network structure obtained by the above network structure adjustment method can be used, but is not limited to, to restore the character image to be restored. Compared with traditional character restoration, the target network structure has the advantage of fast running speed after compression, and also ensures high restoration accuracy.

[0112] It should be noted that the process involves: acquiring the character image to be restored; inputting the character image to be restored into the adjusted target network structure; and acquiring the target restoration result output by the adjusted target network structure, wherein the target restoration result includes the restored character information corresponding to the character image to be restored.

[0113] To further illustrate, the optional based on Figure 3 The scenario shown continues, for example... Figure 6As shown, firstly, a character restoration request triggered on client 602 is obtained. This character restoration request is used to request the restoration of the carried character image. Server 604 receives the character restoration request and uses the adjusted target network structure 302 to perform restoration processing on the character image. The restored character information is then sent to client 602, and client 602 displays the character information in the form of image characters, such as "sunny weather".

[0114] The embodiments provided in this application obtain a character image to be restored; input the character image to be restored into an adjusted target network structure; obtain the target restoration result output by the adjusted target network structure, wherein the target restoration result includes the restored character information corresponding to the character image to be restored, thus achieving the objective and realizing the desired effect.

[0115] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0116] According to another aspect of the present invention, a network structure adjustment apparatus for implementing the above-described network structure adjustment method is also provided. For example... Figure 7 As shown, the device includes:

[0117] Compression unit 702 is used to compress the N convolutional structures included in the target network structure according to M compression methods when the target network structure for restoring the character image is obtained, so as to obtain P candidate network structures, wherein the convolutional structures included in the candidate network structures are N layers, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1.

[0118] The first acquisition unit 704 is used to acquire K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying a sample character to be restored and a second image carrying a restored sample character, and K is a positive integer greater than or equal to 1;

[0119] The restoration unit 706 is used to restore the characters in the first image of each group of sample character images through P candidate network structures, so as to obtain the restored character image corresponding to the first image;

[0120] The comparison unit 708 is used to sequentially compare each piece of restored character information with the character information of the corresponding restored sample character when the restored character information is identified from the restored character image, so as to obtain the character similarity corresponding to the restored character information.

[0121] The adjustment unit 710 is used to replace the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches a similarity threshold.

[0122] Optionally, in this embodiment, the network structure adjustment device can be applied, but is not limited to, in the scenario of text restoration. It determines the candidate network structure with the best performance among candidate network structures with multiple convolutional structure combinations obtained by multiple compression methods, and adjusts the N-layer convolutional structure in the target network structure based on the candidate network structure with the best performance. This ensures that the adjusted target network structure meets the compression requirements while guaranteeing a certain level of restoration accuracy. In other words, by using the above-mentioned network structure adjustment device, the running speed of the target network result can be accelerated without significantly reducing the restoration accuracy of the target network structure, thereby reducing the demand on GPU memory and improving the speed and throughput in practical applications.

[0123] Optionally, in this embodiment, the target network structure may be, but is not limited to, a convolutional neural network structure for restoring character images. The convolutional neural network structure may be, but is not limited to, a type of feedforward neural network that includes convolutional computation and has a deep structure, such as WaveNet network structure, UNet network structure, LeNet-5 network structure, ZFNet network structure, VGGNet network structure, residual neural network structure, etc.

[0124] Optionally, in this embodiment, the target network structure may include, but is not limited to, at least one input structure, at least one output structure, and at least two intermediate structures, wherein the intermediate structures include at least N convolutional structures. Optionally, taking the UNet network structure as an example, the input to the UNet network structure may be, but is not limited to, a 3-channel color smeared image, and the output of the UNet network structure may be, but is not limited to, a 3-channel restored color image.

[0125] Optionally, in this embodiment, since the real-time requirements for the reconstruction of the target network structure are high in some practical applications, it is necessary to compress and accelerate the N-layer convolutional deconstruction of the target network structure while maintaining the reconstruction accuracy as much as possible. For example, Neural Architecture Search (NAS) is used to perform the above compression and acceleration, and a certain number of channels are randomly selected to activate the other layers (e.g., convolutional layers) except for the input and output layers to obtain a series of convolutional layer combinations.

[0126] Furthermore, to limit the number of convolutional layer combinations, the number of channels searchable for each convolutional structure is generally limited. For example, for a convolutional structure with a total of 64 channels, the first 32, 48, or 64 channels can be activated (e.g., randomly). In other words, after performing the above channel number selection on the channel number of each convolutional structure in an N-layer convolutional structure, an N-layer convolutional structure with a certain combination of channel numbers is obtained. Optionally, the compression method can be, but is not limited to, performing the same round of channel number selection on the channel number of each convolutional structure to obtain an N-layer convolutional structure with a new combination of channel numbers, where different compression methods correspond to different combinations of channel numbers.

[0127] Optionally, in this embodiment, assuming there are O types of channel number selection and N layers of convolutional structure, after performing M compression methods on the N-layer convolutional structure, it is possible to obtain, but is not limited to, O to the power of N different combinations of channel number. In addition, taking the UNet network structure as an example, its network structure is a symmetrical U-shaped structure, and the latter half needs to be spliced ​​with the channel features of the former half. Since UNet is symmetrically spliced, if a certain layer in the former half selects a certain number of channels, its symmetrical layer is automatically determined to have twice the number of channels. Therefore, it is possible to obtain, but is not limited to, O to the power of (1 / 2×N) different combinations of channel number.

[0128] Optionally, in this embodiment, each set of sample character images includes a first image and a second image with a corresponding relationship. The second image may be, but is not limited to, the original character image, and the first image may be, but is not limited to, a character image that has been blurred, covered, or smeared based on the original character image. The first image may be, but is not limited to, a clear character that can be identified, and the second image may be, but is not limited to, a clear character that cannot be identified, or a character information that cannot be identified that does not reach the clarity threshold.

[0129] Optionally, in this embodiment, the restored character information can be used, but is not limited to, to represent the character recognition information obtained by character recognition. Character recognition can be, but is not limited to, using optical character recognition (OCR) technology. OCR text recognition can be, but is not limited to, the process by which an electronic device examines a character image and then uses a character recognition device to translate the shape into computer text. That is, the process of scanning text data and then analyzing and processing the image file to obtain text and layout information.

[0130] It should be noted that in the field of text restoration, the restoration effect of the target network model is evaluated using the OCR text recognition results as the evaluation index. The candidate network structure with the best performance is determined, and the target network structure is adjusted to achieve the best network structure that balances real-time restoration and restoration accuracy.

[0131] Optionally, in this embodiment, character similarity can be represented by text edit distance, which can be, but is not limited to, Levenshtein distance, which refers to the minimum number of editing operations required to transform one string or character into another. The larger the distance between them, the lower the similarity; conversely, the smaller the edit distance, the lower the similarity. Permitted editing operations include replacing one character with another, inserting a character, deleting a character, etc.

[0132] It should be noted that, given the target network structure for restoring character images, the N convolutional layers in the target network structure are compressed using M compression methods to obtain P candidate network structures. Each candidate network structure contains N convolutional layers, where N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1. K sets of sample character images are obtained, each set including a first image carrying the character to be restored and a second image carrying the restored character, where K is a positive integer greater than or equal to 1. The first image in each set of sample character images is restored using the P candidate network structures to obtain the restored character image corresponding to the first image. When the restored character information is identified from the restored character image, each restored character information is compared sequentially with the character information of the corresponding restored sample character to obtain the character similarity corresponding to the restored character information. The N convolutional layers in the target network structure are replaced with candidate network structures whose character similarity reaches a similarity threshold.

[0133] Optionally, in this embodiment, since a network structure involving multiple stages is involved, the network structure of these multiple stages can be trained sequentially, but is not limited to. For example, the target network structure can be pre-trained first, and then the adjusted target network structure can be trained a second time. The training samples for pre-training and secondary training can be, but are not limited to, the same or different sample character images. For example, M+N first images and corresponding second images can be obtained. M images (generally M>20000) are used as training data for pre-training, and the remaining N images (generally M>1000) are used as test data for adjusting the target network structure. The training data for secondary training can be freely combined according to needs. For example, M images can be used as training data for secondary training, or M+N images can all be used as training data for secondary training.

[0134] Furthermore, the network structures for these multiple stages can be trained separately, but not limited to, training the target network structure and the adjusted target network structure, or training only the adjusted target network structure, or training only the target network structure. This is merely an example and not a limitation.

[0135] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0136] According to the embodiments provided in this application, when a target network structure for restoring character images is obtained, the N convolutional layers included in the target network structure are compressed using M compression methods to obtain P candidate network structures. Each candidate network structure includes N convolutional layers, where N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1. K sets of sample character images are obtained, where each set of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, where K is a positive integer greater than or equal to 1. The first image in each set of sample character images is restored using the P candidate network structures to obtain a character restoration profile of the first image. The process involves: reconstructing the character image; identifying the reconstructed character information from the reconstructed character image; sequentially comparing each reconstructed character information with the corresponding reconstructed sample character information to obtain the character similarity corresponding to the reconstructed character information; replacing the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold; using character recognition information as an evaluation index in the evaluation process of the compressed convolutional structure in the field of character reconstruction to determine the optimal compressed convolutional structure, thereby adjusting the overall network structure used for character reconstruction, thus achieving the technical objective of ensuring high reconstruction accuracy while meeting the compression requirements of the adjusted network structure, thereby improving the reconstruction accuracy of the compressed network structure.

[0137] As an alternative solution, such as Figure 8 As shown, the compression unit 702 includes:

[0138] Compression module 802 is used to compress N-layer convolutional structures according to M compression methods to obtain M candidate network structures, wherein the candidate network structures include N layers of convolutional structures.

[0139] The first determining module 804 is used to determine P candidate network structures from M candidate network structures whose target computational cost reaches the computational threshold as P candidate network structures, wherein the target computational cost is used to represent the computational cost of one convolution of the candidate network structure.

[0140] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0141] As an optional solution, compression module 902 includes:

[0142] The compression submodule is used to compress the number of channels of each convolutional structure in the N-layer convolutional structure according to M compression methods to obtain M candidate network structures. The channel number sequence corresponding to each candidate network structure is different. The channel number sequence is a sequence synthesized from the channel array of each convolutional structure in the candidate network structure. The number of channels is positively correlated with the target computational cost.

[0143] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0144] As an alternative solution, such as Figure 9 As shown, it includes:

[0145] The second acquisition unit 902 is used to acquire F sets of sample character images before compressing the N layers of convolutional structures included in the target network structure according to M compression methods. Each set of sample character images in the F sets of sample character images includes a first image and a second image, and F is a positive integer greater than or equal to 1.

[0146] The first input unit 904 is used to input F sets of sample character images into the initial target network structure before compressing the N layers of convolutional structures included in the target network structure according to M compression methods, so as to train the target network structure.

[0147] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0148] As an optional solution, it includes:

[0149] The second input unit is used to input K sets of sample character images into the target network structure after inputting F sets of sample character images into the initial target network structure to train the target network structure, so as to train the trained target network structure.

[0150] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0151] As an optional solution, it includes:

[0152] The second input unit is used to input K sets of sample character images into the target network structure after inputting F sets of sample character images into the initial target network structure to train the target network structure, so as to train the trained target network structure.

[0153] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0154] As an optional solution, the comparison unit 708 includes:

[0155] The fourth acquisition module is used to acquire the information difference between each restored character information and the corresponding restored sample character information, wherein the information difference is inversely correlated with the character similarity.

[0156] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0157] As an optional solution, it includes:

[0158] The third acquisition unit is used to acquire the character image to be restored after replacing the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold.

[0159] The third input unit is used to input the character image to be restored into the adjusted target network structure after replacing the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold.

[0160] The fourth acquisition unit is used to acquire the target restoration result output by the adjusted target network structure after replacing the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold. The target restoration result includes the restored character information corresponding to the character image to be restored.

[0161] For specific implementation examples, please refer to the examples shown in the above network structure adjustment method; these will not be repeated here.

[0162] According to another aspect of the present invention, an electronic device for implementing the above-described network structure adjustment method is also provided, such as... Figure 10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps of any of the above method embodiments via the computer program.

[0163] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0164] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0165] S1, Given the target network structure for restoring the character image, compress the N convolutional layers in the target network structure according to M compression methods to obtain P candidate network structures, where the convolutional layers in the candidate network structure are N, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1.

[0166] S2, obtain K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, and K is a positive integer greater than or equal to 1;

[0167] S3, using P candidate network structures to restore the characters in the first image of each group of sample character images, so as to obtain the restored character image corresponding to the first image;

[0168] S4, after recognizing the restored character information from the restored character image, compare each restored character information with the character information of the corresponding restored sample character in turn to obtain the character similarity corresponding to the restored character information;

[0169] S5 replaces the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold.

[0170] Alternatively, as those skilled in the art will understand, Figure 10 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 10 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 10 The different configurations shown.

[0171] The memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the network structure adjustment method and apparatus in this embodiment of the invention. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, thereby realizing the aforementioned network structure adjustment method. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include memory remotely located relative to the processor 1004, and these remote memories can be connected to the terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1002 may be used, but is not limited to, to store information such as sample character images, candidate network structures, and restored character information. As an example, such as... Figure 10 As shown, the memory 1002 may include, but is not limited to, the compression unit 702, the first acquisition unit 704, the restoration unit 706, the comparison unit 708, and the adjustment unit 710 in the network structure adjustment device. Furthermore, it may include, but is not limited to, other module units in the network structure adjustment device, which will not be described in detail in this example.

[0172] Optionally, the transmission device 1006 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1006 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0173] In addition, the above-mentioned electronic device also includes: a display 1008 for displaying the above-mentioned sample character image, candidate network structure and restored character information, etc.; and a connection bus 1010 for connecting the various module components in the above-mentioned electronic device.

[0174] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0175] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the network structure adjustment method described above, wherein the computer program is configured to execute the steps of any of the method embodiments described above during runtime.

[0176] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0177] S1, Given the target network structure for restoring the character image, compress the N convolutional layers in the target network structure according to M compression methods to obtain P candidate network structures, where the convolutional layers in the candidate network structure are N, N is a positive integer greater than or equal to 2, M is a positive integer greater than or equal to 1, and P is a positive integer greater than or equal to 1.

[0178] S2, obtain K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, and K is a positive integer greater than or equal to 1;

[0179] S3, using P candidate network structures to restore the characters in the first image of each group of sample character images, so as to obtain the restored character image corresponding to the first image;

[0180] S4, after recognizing the restored character information from the restored character image, compare each restored character information with the character information of the corresponding restored sample character in turn to obtain the character similarity corresponding to the restored character information;

[0181] S5 replaces the N-layer convolutional structure in the target network structure with a candidate network structure whose character similarity reaches the similarity threshold.

[0182] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0183] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0184] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0185] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for adjusting network structure, characterized in that, include: Given a target network structure for restoring a character image, the number of searchable channels for each of the N convolutional layers in the target network structure is limited by M compression methods to obtain M candidate network structures. Each candidate network structure has a different channel number sequence, which is a sequence synthesized from the channel arrays of each convolutional layer in the candidate network structure. The number of channels is positively correlated with the target computational load. The candidate network structure includes N convolutional layers, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 1. P candidate network structures among the M candidate network structures whose target computational amount reaches the computational threshold are determined as P candidate network structures, wherein the convolutional structure included in the candidate network structure is N layers, and P is a positive integer greater than or equal to 1. Obtain K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying the sample character to be restored and a second image carrying the restored sample character, and K is a positive integer greater than or equal to 1; The first image in each group of sample character images is restored using the P candidate network structures to obtain the restored character image corresponding to the first image. When the restored character information is identified from the restored character image, each restored character information is compared with the character information of the corresponding restored sample character in turn to obtain the character similarity corresponding to the restored character information. Replace the N-layer convolutional structure in the target network structure with the candidate network structure whose character similarity reaches the similarity threshold.

2. The method according to claim 1, characterized in that, Before limiting the number of searchable channels for each of the N convolutional layers in the target network structure according to M compression methods to obtain M candidate network structures, the following steps are included: Obtain F sets of sample character images, wherein each set of sample character images in the F sets of sample character images includes the first image and the second image, and F is a positive integer greater than or equal to 1; The F sets of sample character images are input into the initial target network structure to train and obtain the target network structure.

3. The method according to claim 2, characterized in that, After inputting the F sets of sample character images into the initial target network structure to train the target network structure, the process includes: The K sets of sample character images are input into the target network structure to train and obtain a trained target network structure.

4. The method according to claim 3, characterized in that, The step of inputting the K sets of sample character images into the target network structure to train a trained target network structure includes: Repeat the following steps until the trained target network structure is obtained: The current sample character image is determined from the K groups of sample character images, and the current target network structure is determined, wherein the current sample character image includes the current first image and the current second image; The current restoration result is obtained through the current target network structure, wherein the current restoration result includes the restored character information corresponding to the current first image; If the current restoration result does not meet the convergence condition, the next sample character image is obtained as the current sample character image; If the current restoration result meets the convergence condition, the current target network structure is determined to be the trained target network structure.

5. The method according to any one of claims 1 to 4, characterized in that, The comparison of each restored character information with the corresponding restored sample character information includes: The information difference between each restored character information and the corresponding restored sample character information is obtained, wherein the information difference is inversely correlated with the character similarity.

6. The method according to any one of claims 1 to 4, characterized in that, After replacing the N-layer convolutional structure in the target network structure with the candidate network structure whose character similarity reaches the similarity threshold, the process includes: Obtain the image of the character to be restored; The character image to be restored is input into the adjusted target network structure; Obtain the target restoration result output by the adjusted target network structure, wherein the target restoration result includes the restored character information corresponding to the character image to be restored.

7. A network structure adjustment device, characterized in that, include: A compression unit, upon obtaining a target network structure for reconstructing a character image, restricts the searchable number of channels for each of the N convolutional layers in the target network structure using M compression methods to obtain M candidate network structures. Each candidate network structure has a unique channel number sequence, synthesized from the channel arrays of each convolutional layer in the candidate network structures. The number of channels is positively correlated with the target computational load. The candidate network structures include N convolutional layers, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to 1. P candidate network structures among the M candidate network structures whose target computational load reaches a computational threshold are identified as P candidate network structures, where each candidate network structure includes N convolutional layers, and P is a positive integer greater than or equal to 1. The first acquisition unit is used to acquire K sets of sample character images, wherein each set of sample character images in the K sets of sample character images includes a first image carrying a sample character to be restored and a second image carrying a restored sample character, and K is a positive integer greater than or equal to 1; The restoration unit is used to restore the first image in each group of sample character images through the P candidate network structures to obtain the restored character image corresponding to the first image. The comparison unit is used to sequentially compare each piece of restored character information with the character information of the corresponding restored sample character when the restored character information is identified from the restored character image, so as to obtain the character similarity corresponding to the restored character information; The adjustment unit is used to replace the N-layer convolutional structure in the target network structure with the candidate network structure whose character similarity reaches the similarity threshold.

8. The apparatus according to claim 7, characterized in that, include: The second acquisition unit is used to acquire F sets of sample character images before limiting the number of searchable channels of each convolutional structure in the N convolutional structures included in the target network structure according to M compression methods to obtain M candidate network structures. Each set of sample character images in the F sets of sample character images includes the first image and the second image, and F is a positive integer greater than or equal to 1. The first input unit is used to input the F sets of sample character images into the initial target network structure to train the target network structure before limiting the number of searchable channels of each convolutional structure in the N convolutional structures included in the target network structure according to M compression methods to obtain M candidate network structures.

9. The apparatus according to claim 8, characterized in that, include: The second input unit is used to input the K sets of sample character images into the target network structure after the F sets of sample character images are input into the initial target network structure to train the target network structure, so as to train the trained target network structure.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 6.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 6 through the computer program.

Images