Image processing and visual base model training method and device

Abstract

The invention provides an image processing and visual base model training method and device, and relates to the field of artificial intelligence such as computer vision and deep learning. The image processing method comprises the steps that a target image is acquired, image block segmentation is carried out on the target image, and any two image blocks are not overlapped; and obtaining a two-dimensional rotation position code corresponding to each image block, and generating an image processing result corresponding to the target image by using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to image processing in the fields of computer vision and deep learning, and training methods and devices for visual base models. Background Art

[0002] In practical applications, the length, width, and aspect ratio of an image are important information of the image. When they are forcibly changed, the image content may be distorted and the actual content may be changed. Summary of the invention

[0003] The present disclosure provides a method and device for training an image processing and visual base model.

[0004] An image processing method, comprising:

[0005] Acquire a target image, and divide the target image into image blocks, so that no two image blocks overlap;

[0006] The two-dimensional rotation position codes corresponding to the image blocks are respectively obtained, and according to the image blocks and the corresponding two-dimensional rotation position codes, the image processing results corresponding to the target image are generated by using the visual base model.

[0007] A method for training a visual base model, comprising:

[0008] Acquire training samples, wherein any training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks;

[0009] The visual base model is trained using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0010] An image processing device comprises: a first processing module and a second processing module;

[0011] The first processing module is used to obtain a target image and divide the target image into image blocks, where no two image blocks overlap each other;

[0012] The second processing module is used to respectively obtain the two-dimensional rotation position code corresponding to each image block, and generate the image processing result corresponding to the target image using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

[0013] A training device for a visual base model, comprising: a sample acquisition module and a model training module;

[0014] The sample acquisition module is used to acquire training samples, and any training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks;

[0015] The model training module is used to train the visual base model using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0016] An electronic device, comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0020] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0021] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the above method when executed by a processor.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 A flowchart of an embodiment of the image processing method disclosed in the present invention;

[0025] Figure 2 It is a schematic diagram of the overall implementation process of the image processing method disclosed in the present invention;

[0026] Figure 3 A flowchart of an embodiment of a method for training a visual base model disclosed in the present invention;

[0027] Figure 4 4 is a schematic diagram of the structure of an image processing device embodiment 400 of the present disclosure;

[0028] Figure 5 It is a schematic diagram of the composition structure of an embodiment 500 of the training device for the visual base model disclosed in the present invention;

[0029] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] In addition, it should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0032] Figure 1 FIG. 1 is a flow chart of an embodiment of the image processing method disclosed in the present invention. Figure 1 As shown, the following specific implementation methods are included.

[0033] In step 101, a target image is acquired and segmented into image patches, and no two image patches overlap.

[0034] In step 102, the two-dimensional rotation position codes corresponding to each image block are respectively obtained, and according to each image block and the corresponding two-dimensional rotation position codes, the image processing result corresponding to the target image is generated using the visual base model.

[0035] The base model can serve as the basis for other models and can be fine-tuned on its basis to adapt to tasks in different specific scenarios. The visual base model refers to the base model used to complete visual processing related tasks, such as the contrastive language-image pre-training model (CLIP) using contrastive learning and the masked autoencoder model (MAE).

[0036] In the traditional method, the input image is usually directly adjusted to a square of a predetermined size, such as 512*512, 336*336, etc., and then the adjusted image is divided into multiple image blocks, each of which is also a square, such as 32*32, 14*14, etc., and the adjusted image size is an integer multiple of the image block size, and then each image block is respectively encoded with a one-dimensional absolute position, and the required image processing results are generated using the visual base model according to each image block and the corresponding one-dimensional absolute position code. Since the image needs to be forcibly adjusted to a square of a predetermined size, it will inevitably cause the loss of image information, that is, the degradation of image quality, and thus reduce the accuracy of the obtained image processing results.

[0037] By adopting the scheme described in the above method embodiment, each image block cut out from the target image can be respectively encoded with a two-dimensional rotation position, that is, the two-dimensional position information of each image block can be respectively represented, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0038] In some embodiments of the present disclosure, the image block obtained by segmentation may include M pixels, where M is a positive integer greater than one, and the image block may be a square-shaped image block. Accordingly, the method for obtaining a target image may include: obtaining an image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length in the positive direction, and the width of the image to be processed is an integer multiple of the side length in the positive direction, taking the image to be processed as the target image, and in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image.

[0039] The specific value of M can be determined according to actual needs, such as 100, that is, the image blocks obtained by segmentation can be image blocks of size 10*10.

[0040] That is to say, after obtaining the image to be processed, its size can be judged first. If it is determined that the image to be processed meets the requirement of "the length is an integer multiple of the side length in the positive direction, and the width is an integer multiple of the side length in the positive direction", then the image to be processed can be directly used as the target image. For example, if the image to be processed is 610*520 in size, then it can be determined that the image to be processed meets the requirements, and thus the image to be processed can be directly used as the target image. If it is determined that the image to be processed does not meet the requirements, such as the image to be processed is 612*518 in size, then the image to be processed can be adjusted to meet the requirements in accordance with the principle of minimizing the impact on the image quality of the image to be processed, such as adjusting it to 610*520. It can be seen that this adjustment method will only make a very small adjustment to the length and width of the image to be processed, so as not to have a significant impact on the image quality of the image to be processed. Further, the adjusted image to be processed can be used as the target image.

[0041] Assuming that the image block is 10*10 in size, for a target image of 610*520 in size, it can be divided into 61*52 image blocks.

[0042] In the above description, each image block includes a plurality of pixels. In practical applications, each image block may include one pixel, that is, when the image to be processed is divided into image blocks, each pixel in the image to be processed may be used as an image block. In this case, no matter what size the image to be processed is, it is usually not necessary to adjust it.

[0043] For each image block, the corresponding two-dimensional rotation position code can be obtained respectively. In some embodiments of the present disclosure, the target beta (beta, β) value corresponding to the target image can be determined first, and then the following processing can be performed for any image block: the x-direction position and the y-direction position of the image block are beta-encoded (or called beta coding) according to the target beta value, and the obtained x-direction coding results and y-direction coding results are used as the two-dimensional rotation position code of the image block.

[0044] For example, assuming that the target image of size 610*520 is divided into 61*52 image blocks, then for any image block, as a simple way of understanding, the row where the image block is located can be used as the x-direction position coordinate, and the column where the image block is located can be used as the y-direction position coordinate. In addition, the x-direction position coordinate and the y-direction position coordinate can be beta-encoded according to the determined target beta value, thereby obtaining the x-direction encoding result and the y-direction encoding result of the image block, which together constitute the two-dimensional rotational position code of the image block.

[0045] Beta encoding is an existing encoding method based on the idea of ​​base conversion. It understands position encoding by analogy with base. That is, by encoding position information into base form, the model can better understand and process the position relationship in the text, especially when processing long texts, which can significantly improve the performance of the model.

[0046] In addition, due to the use of two-dimensional rotational position encoding, the position coordinates of each image block enter the model in a two-dimensional manner rather than the traditional one-dimensional manner, so it can adapt to target images of arbitrary aspect ratios.

[0047] In some embodiments of the present disclosure, a method for determining a target beta value corresponding to a target image may include: in response to determining that the length and width of the target image are both less than or equal to a first value, using the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

[0048] Generally speaking, the training beta value used in training the visual base model is fixed, and the specific value can be determined according to actual needs, such as the size range of the training samples used in training.

[0049] Assuming that the maximum value of the length and width of each training image is 1000, and the length of the target image is 610 and the width is 520, it means that the length and width of the target image are both less than or equal to the first value. Accordingly, the training beta value can be used as the target beta value corresponding to the target image.

[0050] That is to say, in the above case, the training beta value can be used directly, so that the required two-dimensional rotation position encoding can be obtained quickly and accurately, laying a good foundation for subsequent processing.

[0051] In some embodiments of the present disclosure, in response to determining that at least one of the length and width of the target image is greater than a first value, the target beta value can be determined according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

[0052] If it is determined that at least one of the length and width of the target image is greater than the first value, then the target image can be considered to be a high-resolution image, and the training beta value can be adjusted according to the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, thereby obtaining the target beta value.

[0053] That is, by adjusting the beta value used in beta encoding, the values ​​of the x-direction encoding results and the y-direction encoding results corresponding to the target image can be made to be within the trained (i.e., learned) value range, thereby ensuring that the processing of the visual base model is not affected. Moreover, the visual base model can have length extrapolation, thereby supporting arbitrary high-resolution image input. Length extrapolation means that low-resolution images are trained during training, while high-resolution images are acceptable during inference.

[0054] In addition, in some embodiments of the present disclosure, the larger value of the length and width of the target image can be used as the second value, and the target beta value can be calculated by a predetermined transformation formula based on the second value, the first value, and the training beta value. The specific form of the predetermined transformation formula is not limited.

[0055] In addition, other methods may be used to determine the target beta value. For example, the second value, the first value, and the training beta value may be used as inputs of a pre-trained evaluation model to obtain an output target beta value.

[0056] Assuming that the value range during training is [a, b], and b is greater than a, then after the above processing, when the x-direction position coordinates and y-direction position coordinates of each image block in the target image are beta-encoded according to the obtained target beta value, the obtained x-direction encoding results and y-direction encoding results will both be within the value range [a, b].

[0057] After respectively acquiring the two-dimensional rotation position codes corresponding to the image blocks, the image processing results corresponding to the target image can be generated using the visual base model according to the image blocks and the corresponding two-dimensional rotation position codes.

[0058] Combined with the above introduction, Figure 2 FIG. 1 is a schematic diagram of the overall implementation process of the image processing method disclosed in the present invention. Figure 2As shown, for the image to be processed, assuming that it does not meet the following requirements: the length of the image to be processed is an integer multiple of the side length of the image block, and the width of the image to be processed is an integer multiple of the side length of the image block, and the image block is in the shape of a square, then image adjustment processing can be performed, that is, the image to be processed can be adjusted to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image. After that, image segmentation processing and image coding processing can be performed in sequence, that is, the target image can be segmented into multiple image blocks, and the two-dimensional rotation position code corresponding to each image block can be obtained respectively, wherein the target beta value corresponding to the target image can be determined, and the following processing can be performed for any image block: the x-direction position coordinates and the y-direction position coordinates of the image block are beta-coded respectively according to the target beta value, and the obtained x-direction coding results and y-direction coding results are used as the two-dimensional rotation position code of the image block. Further, according to each image block and the corresponding two-dimensional rotation position code, the image processing result corresponding to the target image can be generated using the visual base model.

[0059] The visual base model can be pre-trained. Figure 3 FIG. 1 is a flow chart of an embodiment of a method for training a visual base model according to the present disclosure. Figure 3 As shown, the following specific implementation methods are included.

[0060] In step 301, a training sample is obtained, and any training sample includes: image blocks obtained by segmenting a training image and two-dimensional rotation position codes corresponding to the image blocks.

[0061] In step 302, the visual base model is trained using training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0062] By adopting the scheme described in the above method embodiment, a visual base model can be trained, and then the visual base model can be used for actual reasoning applications (such as application to Figure 1 In the embodiment shown in the figure), each image block cut out from the target image can be respectively encoded with a two-dimensional rotation position, and the visual base model can be used to generate an image processing result corresponding to the target image according to each image block and the two-dimensional rotation position encoding corresponding to each image block, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0063] In addition, in some embodiments of the present disclosure, the image block may include M pixels, M is a positive integer greater than one, and the image block is a square-shaped image block. Accordingly, the training image needs to meet the following requirements: the length of the training image is an integer multiple of the side length in the positive direction, and the width of the training image is an integer multiple of the side length in the positive direction. Similarly, the target image also needs to meet the following requirements: the length of the target image is an integer multiple of the side length in the positive direction, and the width of the target image is an integer multiple of the side length in the positive direction.

[0064] Accordingly, if the training image or the target image does not meet the above requirements, its size may be adjusted first to meet the requirements, thereby facilitating subsequent segmentation of image blocks.

[0065] In addition, if necessary, the training samples may also include corresponding labels, such as image classification result labels (such as classification into cat, dog, etc.), image segmentation result labels, mask prediction result labels, etc.

[0066] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0067] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0068] Figure 4 FIG. 4 is a schematic diagram of the structure of an image processing device embodiment 400 of the present disclosure. Figure 4 As shown, it includes: a first processing module 401 and a second processing module 402.

[0069] The first processing module 401 is used to obtain a target image and divide the target image into image blocks, and there is no overlap between any two image blocks.

[0070] The second processing module 402 is used to respectively obtain the two-dimensional rotation position code corresponding to each image block, and generate an image processing result corresponding to the target image using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

[0071] By adopting the scheme described in the above-mentioned device embodiment, each image block cut out from the target image can be respectively encoded with a two-dimensional rotation position, that is, the two-dimensional position information of each image block can be respectively represented, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0072] In some embodiments of the present disclosure, the image block obtained by segmentation may include M pixels, where M is a positive integer greater than one, and the image block may be a square-shaped image block. Accordingly, the method for the first processing module 401 to obtain the target image may include: obtaining the image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length in the positive direction, and the width of the image to be processed is an integer multiple of the side length in the positive direction, taking the image to be processed as the target image, and in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image.

[0073] For each image block, the corresponding two-dimensional rotation position code can be obtained respectively. In some embodiments of the present disclosure, the second processing module 402 can first determine the target beta value corresponding to the target image, and then can perform the following processing for any image block: Beta code the x-direction position and y-direction position of the image block according to the target beta value, and use the obtained x-direction coding result and y-direction coding result as the two-dimensional rotation position code of the image block.

[0074] In some embodiments of the present disclosure, the way in which the second processing module 402 determines the target beta value corresponding to the target image may include: in response to determining that the length and width of the target image are both less than or equal to a first value, using the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

[0075] In addition, in some embodiments of the present disclosure, in response to determining that at least one of the length and the width of the target image is greater than the first value, the second processing module 402 may determine the target beta value according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

[0076] Specifically, in some embodiments of the present disclosure, the second processing module 402 may take the larger value of the length and width of the target image as the second value, and may calculate the target beta value through a predetermined transformation formula based on the second value, the first value and the training beta value.

[0077] Figure 5 FIG. 5 is a schematic diagram of the structure of a training device embodiment 500 of a visual base model according to the present disclosure. Figure 5 As shown, it includes: a sample acquisition module 501 and a model training module 502.

[0078] The sample acquisition module 501 is used to acquire training samples. Any training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks.

[0079] The model training module 502 is used to train the visual base model using training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0080] By adopting the scheme described in the above-mentioned device embodiment, a visual base model can be trained, and then the visual base model can be used for actual reasoning applications, that is, each image block cut out from the target image can be respectively encoded with a two-dimensional rotation position, and the visual base model can be used to generate an image processing result corresponding to the target image according to each image block and the two-dimensional rotation position encoding corresponding to each image block, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0081] In addition, in some embodiments of the present disclosure, the image block may include M pixels, M is a positive integer greater than one, and the image block is a square image block. Accordingly, the training image needs to meet the following requirements: the length of the training image is an integer multiple of the side length in the positive direction, and the width of the training image is an integer multiple of the side length in the positive direction. Similarly, the target image also needs to meet the following requirements: the length of the target image is an integer multiple of the side length in the positive direction, and the width of the target image is an integer multiple of the side length in the positive direction.

[0082] Figure 4 and Figure 5 The specific working process of the illustrated device embodiment can refer to the relevant description in the aforementioned method embodiment and will not be described in detail.

[0083] The scheme disclosed in the present invention can be applied to the field of artificial intelligence, especially computer vision and deep learning. Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0084] The images and the like in the embodiments described in this disclosure are not targeted at a specific user and do not reflect the personal information of a specific user. In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0086] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0087] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0088] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0089] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI, Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP, Digital Signal Processing), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the methods described in the present disclosure may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method described in the present disclosure in any other appropriate manner (for example, by means of firmware).

[0090] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0092] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, Electronically Programmable Read-Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM, Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0095] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Acquire a target image, and divide the target image into image blocks, so that no two image blocks overlap; The two-dimensional rotation position codes corresponding to the image blocks are respectively obtained, and according to the image blocks and the corresponding two-dimensional rotation position codes, the image processing results corresponding to the target image are generated by using the visual base model.

2. The method according to claim 1, wherein: The image block includes M pixels, M is a positive integer greater than one, and the image block is a square image block; The acquiring of the target image comprises: Acquire an image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length in the positive direction, and the width of the image to be processed is an integer multiple of the side length in the positive direction, use the image to be processed as the target image; In response to determining that the image to be processed does not meet the requirement, the image to be processed is adjusted to meet the requirement according to the principle of minimizing the impact on the image quality of the image to be processed, so as to obtain the target image.

3. The method according to claim 1 or 2, wherein: The step of respectively acquiring the two-dimensional rotation position codes corresponding to each image block comprises: Determining a target beta value corresponding to the target image; For any image block, the following processing is performed respectively: the x-direction position coordinates and the y-direction position coordinates of the image block are beta-encoded according to the target beta value, and the obtained x-direction encoding results and y-direction encoding results are used as the two-dimensional rotation position encoding of the image block.

4. The method according to claim 3, wherein: Determining the target beta value corresponding to the target image includes: In response to determining that the length and width of the target image are both less than or equal to a first value, the training beta value used when training the visual base model is used as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

5. The method according to claim 4, further comprising: In response to determining that at least one of the length and the width of the target image is greater than the first value, the target beta value is determined according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

6. The method according to claim 5, wherein: Determining the target beta value according to the training beta value includes: The larger value of the length and the width of the target image is used as a second value, and the target beta value is calculated using a predetermined transformation formula according to the second value, the first value and the training beta value.

7. A method for training a visual base model, comprising: Acquire training samples, wherein any training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks; The visual base model is trained using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

8. The method according to claim 7, wherein: The image block includes M pixels, M is a positive integer greater than one, and the image block is a square image block; The training image meets the following requirements: the length of the training image is an integer multiple of the length of the side in the positive direction, and the width of the training image is an integer multiple of the length of the side in the positive direction; The target image meets the following requirements: the length of the target image is an integer multiple of the side length in the positive direction, and the width of the target image is an integer multiple of the side length in the positive direction.

9. An image processing device, comprising: A first processing module and a second processing module; The first processing module is used to obtain a target image and divide the target image into image blocks, where no two image blocks overlap each other; The second processing module is used to respectively obtain the two-dimensional rotation position code corresponding to each image block, and generate the image processing result corresponding to the target image using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

10. The device according to claim 9, wherein: The image block includes M pixels, M is a positive integer greater than one, and the image block is a square image block; The first processing module acquires an image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length in the positive direction, and the width of the image to be processed is an integer multiple of the side length in the positive direction, takes the image to be processed as the target image, and in response to determining that the image to be processed does not meet the requirements, adjusts the image to be processed to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image.

11. The device according to claim 9 or 10, wherein: The second processing module determines the target beta value corresponding to the target image, and performs the following processing for any image block: Beta-encodes the x-direction position coordinates and the y-direction position coordinates of the image block according to the target beta value, and uses the obtained x-direction encoding results and y-direction encoding results as the two-dimensional rotation position encoding of the image block.

12. The device according to claim 11, wherein In response to determining that the length and width of the target image are both less than or equal to a first value, the second processing module uses the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

13. The device according to claim 12, wherein: The second processing module is further used to, in response to determining that at least one of the length and the width of the target image is greater than the first value, determine the target beta value according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

14. The device according to claim 13, wherein: The second processing module uses the larger value of the length and width of the target image as a second value, and calculates the target beta value through a predetermined transformation formula according to the second value, the first value and the training beta value.

15. A training device for a visual base model, comprising: Sample acquisition module and model training module; The sample acquisition module is used to acquire training samples, and any training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks; The model training module is used to train the visual base model using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

16. The device according to claim 15, wherein: The image block includes M pixels, M is a positive integer greater than one, and the image block is a square image block; The training image meets the following requirements: the length of the training image is an integer multiple of the length of the side in the positive direction, and the width of the training image is an integer multiple of the length of the side in the positive direction; The target image meets the following requirements: the length of the target image is an integer multiple of the side length in the positive direction, and the width of the target image is an integer multiple of the side length in the positive direction.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images