Image processing and visual base model training method and device

Abstract

This disclosure provides methods and devices for image processing and training a visual pedestal model, relating to artificial intelligence fields such as computer vision and deep learning. The image processing method may include: acquiring a target image and segmenting the target image into image blocks, with no overlap between any two image blocks; obtaining a two-dimensional rotational position code corresponding to each image block; and generating an image processing result corresponding to the target image using a visual pedestal model based on each image block and the corresponding two-dimensional rotational position code.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to image processing and visual base model training methods and devices in the fields of computer vision and deep learning. Background Art

[0002] In practical applications, the length, width, and aspect ratio of an image are important information of the image. When they are forcibly changed, the image content may be distorted or the actual content may be changed. Summary of the Invention

[0003] The present disclosure provides a method and apparatus for image processing and training a visual base model.

[0004] An image processing method, comprising:

[0005] Obtain a target image and segment the target image into image blocks, so that no two image blocks overlap;

[0006] The two-dimensional rotation position code corresponding to each image block is obtained respectively, and the image processing result corresponding to the target image is generated using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

[0007] A method for training a visual base model, comprising:

[0008] Acquire training samples, where each training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks;

[0009] The visual base model is trained using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0010] An image processing device comprises: a first processing module and a second processing module;

[0011] The first processing module is used to obtain a target image and divide the target image into image blocks, so that no two image blocks overlap;

[0012] The second processing module is used to respectively obtain the two-dimensional rotation position code corresponding to each image block, and generate an image processing result corresponding to the target image using a visual base model based on each image block and the corresponding two-dimensional rotation position code.

[0013] A training device for a visual base model, comprising: a sample acquisition module and a model training module;

[0014] The sample acquisition module is used to acquire training samples, each training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks;

[0015] The model training module is used to train the visual base model using the training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0016] An electronic device, comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0020] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0021] A computer program product comprises a computer program / instruction, which implements the above method when executed by a processor.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0024] Figure 1 This is a flowchart of an embodiment of the image processing method disclosed herein;

[0025] Figure 2 Schematic diagram of the overall implementation process of the image processing method disclosed in the present invention;

[0026] Figure 3 This is a flowchart of an embodiment of a method for training a visual base model according to the present disclosure;

[0027] Figure 4 Schematic diagram of the structure of an image processing device embodiment 400 of the present disclosure;

[0028] Figure 5 Schematic diagram of the structure of a training device embodiment 500 of a visual base model according to the present disclosure;

[0029] Figure 6 FIG. 6 is a schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] Furthermore, it should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " as used herein generally indicates that the associated objects are in an "or" relationship.

[0032] Figure 1 Flowchart of the image processing method embodiment of the present disclosure. Figure 1 As shown, the following specific implementation methods are included.

[0033] In step 101 , a target image is acquired and divided into image patches, with no overlap between any two image patches.

[0034] In step 102, the two-dimensional rotation position code corresponding to each image block is obtained respectively, and the image processing result corresponding to the target image is generated using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

[0035] The base model can serve as the foundation for other models and can be fine-tuned to adapt to tasks in different specific scenarios. The visual base model refers to the base model used to complete visual processing-related tasks, such as the text-image pre-training model using contrastive learning (CLIP) and the masked autoencoder model (MAE).

[0036] Traditional methods typically resize the input image to a predetermined square size, such as 512*512 or 336*336. The resized image is then divided into multiple image blocks, each also a square, such as 32*32 or 14*14. The resized image is an integer multiple of the block size. Each block is then individually coded for one-dimensional absolute position. Based on each block and its corresponding one-dimensional absolute position code, a visual pedestal model is used to generate the desired image processing results. This forced resizing of the image to a predetermined square size inevitably results in a loss of image information, which in turn reduces image quality and the accuracy of the resulting image processing results.

[0037] By adopting the scheme described in the above method embodiment, each image block cut out from the target image can be respectively encoded with two-dimensional rotation position, and the two-dimensional position information of each image block can be respectively represented, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0038] In some embodiments of the present disclosure, the image block obtained by segmentation may include M pixels, where M is a positive integer greater than one, and the image block may be a square-shaped image block. Accordingly, the method of obtaining the target image may include: obtaining the image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length of the square, and the width of the image to be processed is an integer multiple of the side length of the square, using the image to be processed as the target image, and in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image.

[0039] The specific value of M can be determined according to actual needs. For example, 100 means that the image blocks obtained by segmentation can be image blocks of size 10*10.

[0040] That is to say, after obtaining the image to be processed, its size can be judged first. If it is determined that the image to be processed meets the requirement of "the length is an integer multiple of the side length of the square, and the width is an integer multiple of the side length of the square", then the image to be processed can be directly used as the target image. For example, if the image to be processed is 610*520 in size, then it can be determined that the image to be processed meets the requirements, and thus the image to be processed can be directly used as the target image. If it is determined that the image to be processed does not meet the requirements, such as the image to be processed is 612*518 in size, then the image to be processed can be adjusted to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, such as adjusting it to 610*520. It can be seen that this adjustment method will only make a very small adjustment to the length and width of the image to be processed, so as not to have a large impact on the image quality of the image to be processed. Furthermore, the adjusted image to be processed can be used as the target image.

[0041] Assuming that the image block size is 10*10, then for a target image of size 610*520, it can be divided into 61*52 image blocks.

[0042] The above explanation uses the example of each image block containing multiple pixels. In actual applications, each image block can also contain a single pixel. That is, when dividing the image to be processed into blocks, each pixel in the image to be processed can be considered as an image block. In this case, no adjustment is usually required regardless of the size of the image to be processed.

[0043] For each image block, a corresponding two-dimensional rotational position code can be obtained. In some embodiments of the present disclosure, a target beta (β) value corresponding to the target image can be first determined. Then, for each image block, the following processing can be performed: the x- and y-direction positions of the image block are beta-encoded (or beta-encoded) based on the target beta value, and the obtained x- and y-direction encoding results are used as the two-dimensional rotational position code for the image block.

[0044] For example, assuming that a target image of size 610*520 is divided into 61*52 image blocks, then for any image block, as a simple way of understanding, the row where the image block is located can be used as the x-direction position coordinate, and the column where the image block is located can be used as the y-direction position coordinate. According to the determined target beta value, the x-direction position coordinate and the y-direction position coordinate can be beta-encoded respectively, thereby obtaining the x-direction encoding result and the y-direction encoding result of the image block, which together constitute the two-dimensional rotational position code of the image block.

[0045] Beta encoding is an existing encoding method based on the idea of ​​base conversion. It understands positional encoding by analogy with base. That is, by encoding position information into base form, the model can better understand and process positional relationships in the text. This can significantly improve model performance, especially when processing long texts.

[0046] In addition, due to the use of two-dimensional rotation position encoding, the position coordinates of each image block enter the model in a two-dimensional manner instead of the traditional one-dimensional manner, so it can adapt to target images of arbitrary aspect ratios.

[0047] In some embodiments of the present disclosure, a method for determining a target beta value corresponding to a target image may include: in response to determining that the length and width of the target image are both less than or equal to a first value, using the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

[0048] Generally speaking, the training beta value used in training the visual base model is fixed, and the specific value can be determined according to actual needs, such as the size range of the training samples used in training.

[0049] Assuming that the maximum value of the length and width of each training image is 1000, and the length of the target image is 610 and the width is 520, then it means that the length and width of the target image are both less than or equal to the first value. Accordingly, the training beta value can be used as the target beta value corresponding to the target image.

[0050] That is to say, in the above case, the training beta value can be directly used to quickly and accurately obtain the required two-dimensional rotation position encoding, laying a good foundation for subsequent processing.

[0051] In some embodiments of the present disclosure, in response to determining that at least one of the length and width of the target image is greater than a first value, the target beta value can be determined according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range. The predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

[0052] If it is determined that at least one of the length and width of the target image is greater than the first value, then the target image can be considered to be a high-resolution image, and the training beta value can be adjusted according to the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within the predetermined value range, thereby obtaining the target beta value.

[0053] That is, by adjusting the beta value used in beta encoding, the values ​​of the x-direction encoding results and the y-direction encoding results corresponding to the target image can be made to be within the trained (i.e., learned) value range, thereby ensuring that the processing of the visual base model is not affected. Moreover, the visual base model can have length extrapolation, thereby supporting arbitrary high-resolution image input. Length extrapolation means that low-resolution images are trained during training, while high-resolution images can be accepted during inference.

[0054] In addition, in some embodiments of the present disclosure, the larger of the length and width of the target image can be used as the second value, and the target beta value can be calculated using a predetermined transformation formula based on the second value, the first value, and the training beta value. The specific form of the predetermined transformation formula is not limited.

[0055] In addition, other methods can be used to determine the target beta value. For example, the second value, the first value, and the training beta value can be used as inputs of a pre-trained evaluation model to obtain the output target beta value.

[0056] Assuming that the value range during training is [a, b], and b is greater than a, then after the above processing, when the x-direction position coordinates and y-direction position coordinates of each image block in the target image are beta-encoded according to the obtained target beta value, the obtained x-direction encoding results and y-direction encoding results will both be within the value range [a, b].

[0057] After respectively obtaining the two-dimensional rotation position codes corresponding to the image blocks, the image processing results corresponding to the target image can be generated using the visual base model according to the image blocks and the corresponding two-dimensional rotation position codes.

[0058] Combined with the above introduction, Figure 2 Schematic diagram of the overall implementation process of the image processing method disclosed in this disclosure. Figure 2As shown, for an image to be processed, assuming it does not meet the following requirements: the length of the image to be processed is an integer multiple of the side length of the image block, the width of the image to be processed is an integer multiple of the side length of the image block, and the image block is square, image adjustment processing can be performed. Specifically, the image to be processed can be adjusted to meet the requirements based on the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining a target image. Subsequently, image segmentation and image encoding processing can be performed sequentially, namely, the target image can be segmented into multiple image blocks, and the corresponding two-dimensional rotational position code for each image block can be obtained. A target beta value corresponding to the target image can be determined, and the following processing can be performed on each image block: the x-direction position coordinates and y-direction position coordinates of the image block are beta-encoded according to the target beta value, and the obtained x-direction encoding results and y-direction encoding results are used as the two-dimensional rotational position code for the image block. Furthermore, based on each image block and the corresponding two-dimensional rotational position code, the visual base model can be used to generate the image processing result corresponding to the target image.

[0059] The visual base model can be pre-trained. Figure 3 Flowchart of an embodiment of the training method of the visual base model disclosed in the present invention. Figure 3 As shown, the following specific implementation methods are included.

[0060] In step 301, a training sample is obtained. Any training sample includes: image blocks obtained by segmenting a training image and two-dimensional rotation position codes corresponding to the image blocks.

[0061] In step 302, the visual base model is trained using training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0062] By adopting the scheme described in the above method embodiment, a visual base model can be trained and then used to perform practical reasoning applications (such as applying to Figure 1 In the embodiment shown), each image block cut out from the target image can be respectively subjected to two-dimensional rotation position encoding, and the visual base model can be used to generate an image processing result corresponding to the target image according to each image block and the two-dimensional rotation position encoding corresponding to each image block, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing result.

[0063] In addition, in some embodiments of the present disclosure, the image block may include M pixels, where M is a positive integer greater than one, and the image block is a square-shaped image block. Accordingly, the training image needs to meet the following requirements: the length of the training image is an integer multiple of the side length of the square, and the width of the training image is an integer multiple of the side length of the square. Similarly, the target image also needs to meet the following requirements: the length of the target image is an integer multiple of the side length of the square, and the width of the target image is an integer multiple of the side length of the square.

[0064] Accordingly, if the training image or the target image does not meet the above requirements, its size may be adjusted first to meet the requirements, thereby facilitating subsequent segmentation of image blocks.

[0065] In addition, if necessary, the training samples may also include corresponding labels, such as image classification result labels (such as classification into cat, dog, etc.), image segmentation result labels, mask prediction result labels, etc.

[0066] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0067] The above is an introduction to the method embodiment. The following is a further explanation of the solution disclosed in the present disclosure through an apparatus embodiment.

[0068] Figure 4 FIG. 4 is a schematic diagram of the structure of an image processing device embodiment 400 of the present disclosure. Figure 4 As shown, it includes: a first processing module 401 and a second processing module 402.

[0069] The first processing module 401 is used to obtain a target image and divide the target image into image blocks, so that no two image blocks overlap.

[0070] The second processing module 402 is configured to obtain the two-dimensional rotation position code corresponding to each image block, and generate an image processing result corresponding to the target image using the visual base model according to each image block and the corresponding two-dimensional rotation position code.

[0071] By adopting the scheme described in the above-mentioned device embodiment, each image block cut out from the target image can be respectively subjected to two-dimensional rotation position encoding, that is, the two-dimensional position information of each image block can be respectively represented, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0072] In some embodiments of the present disclosure, the image block obtained by segmentation may include M pixels, where M is a positive integer greater than one, and the image block may be a square-shaped image block. Accordingly, the way in which the first processing module 401 obtains the target image may include: obtaining the image to be processed, and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length of the square, and the width of the image to be processed is an integer multiple of the side length of the square, taking the image to be processed as the target image, and in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements according to the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image.

[0073] For each image block, a corresponding two-dimensional rotational position code can be obtained. In some embodiments of the present disclosure, the second processing module 402 can first determine a target beta value corresponding to the target image, and then perform the following processing on each image block: beta-encode the x-direction position and y-direction position of the image block according to the target beta value, and use the obtained x-direction encoding results and y-direction encoding results as the two-dimensional rotational position code of the image block.

[0074] In some embodiments of the present disclosure, the way in which the second processing module 402 determines the target beta value corresponding to the target image may include: in response to determining that the length and width of the target image are both less than or equal to a first value, using the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

[0075] In addition, in some embodiments of the present disclosure, in response to determining that at least one of the length and width of the target image is greater than the first value, the second processing module 402 may determine the target beta value based on the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

[0076] Specifically, in some embodiments of the present disclosure, the second processing module 402 may take the larger value of the length and width of the target image as the second value, and may calculate the target beta value through a predetermined transformation formula based on the second value, the first value and the training beta value.

[0077] Figure 5 FIG. 5 is a schematic diagram of the structure of a training device embodiment 500 for a visual base model according to the present disclosure. Figure 5 As shown, it includes: a sample acquisition module 501 and a model training module 502.

[0078] The sample acquisition module 501 is used to acquire training samples. Each training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to the image blocks.

[0079] The model training module 502 is used to train the visual base model using training samples. The visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position encoding corresponding to each image block. There is no overlap between any two image blocks corresponding to the same image.

[0080] By adopting the scheme described in the above-mentioned device embodiment, a visual base model can be trained and obtained, and then the visual base model can be used for actual reasoning applications, that is, each image block cut out from the target image can be respectively encoded with a two-dimensional rotation position, and the visual base model can be used to generate an image processing result corresponding to the target image according to each image block and the two-dimensional rotation position encoding corresponding to each image block, thereby eliminating the need to forcibly adjust the target image to a square of a predetermined size, thereby reducing the loss of image information and improving the accuracy of the obtained image processing results.

[0081] In addition, in some embodiments of the present disclosure, the image block may include M pixels, where M is a positive integer greater than one, and the image block is a square image block. Accordingly, the training image needs to meet the following requirements: the length of the training image is an integer multiple of the side length of the square, and the width of the training image is an integer multiple of the side length of the square. Similarly, the target image also needs to meet the following requirements: the length of the target image is an integer multiple of the side length of the square, and the width of the target image is an integer multiple of the side length of the square.

[0082] Figure 4 and Figure 5 The specific working process of the device embodiment shown can refer to the relevant description in the aforementioned method embodiment and will not be repeated here.

[0083] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly computer vision and deep learning. Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0084] The images and other information in the embodiments described in this disclosure are not targeted at any specific user and do not reflect any personal information about any specific user. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in the technical solutions of this disclosure comply with relevant laws and regulations and do not violate public order and good morals.

[0085] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0086] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0087] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. Computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0088] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0089] The computing unit 601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the methods described herein. For example, in some embodiments, the methods described herein may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the methods described herein may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method described in the present disclosure in any other appropriate manner (for example, by means of firmware).

[0090] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0095] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0096] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0097] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image processing method, comprising: Acquire a target image and divide the target image into image blocks, wherein no two image blocks overlap; wherein the image blocks include M pixels, M is a positive integer greater than one, and the image blocks are square-shaped; acquiring the target image includes: acquiring an image to be processed; in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length of the square, and the width of the image to be processed is an integer multiple of the side length of the square, using the image to be processed as the target image; in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements based on the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image; Obtaining the two-dimensional rotational position code corresponding to each image block respectively includes: determining a target beta value corresponding to the target image, and performing the following processing on any image block: beta-encoding the x-direction position coordinates and the y-direction position coordinates of the image block according to the target beta value, using the obtained x-direction encoding results and the y-direction encoding results as the two-dimensional rotational position code of the image block, and generating an image processing result corresponding to the target image using a visual base model based on each image block and the corresponding two-dimensional rotational position code.

2. The method according to claim 1, wherein Determining the target beta value corresponding to the target image includes: In response to determining that the length and width of the target image are both less than or equal to a first value, the training beta value used when training the visual base model is used as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

3. The method according to claim 2, further comprising: In response to determining that at least one of the length and the width of the target image is greater than the first value, the target beta value is determined according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, and the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

4. The method according to claim 3, wherein: Determining the target beta value according to the training beta value includes: The larger value of the length and width of the target image is used as the second value, and the target beta value is calculated using a predetermined transformation formula based on the second value, the first value and the training beta value.

5. A method for training a visual base model, comprising: Obtaining training samples, each training sample comprising: image blocks obtained by segmenting a training image and two-dimensional rotational position codes corresponding to each image block; wherein the two-dimensional rotational position codes corresponding to each image block comprise: an x-direction encoding result and a y-direction encoding result obtained by beta-encoding the x-direction position coordinates and the y-direction position coordinates of the image block according to the determined target beta value; The visual base model is trained using the training samples, wherein the visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to the image blocks, wherein no two image blocks corresponding to the same image overlap; The image block includes M pixels, M is a positive integer greater than one, and the image block is a square-shaped image block; The training image meets the following requirements: the length of the training image is an integer multiple of the side length of the square, and the width of the training image is an integer multiple of the side length of the square; The target image meets the following requirements: the length of the target image is an integer multiple of the side length of the square, and the width of the target image is an integer multiple of the side length of the square.

6. An image processing device comprising: a first processing module and a second processing module; The first processing module is configured to acquire a target image and divide the target image into image blocks, wherein no two image blocks overlap; wherein the image blocks include M pixels, where M is a positive integer greater than one, and the image blocks are square-shaped; acquiring the target image comprises: acquiring an image to be processed; and in response to determining that the image to be processed meets the following requirements: the length of the image to be processed is an integer multiple of the side length of the square, and the width of the image to be processed is an integer multiple of the side length of the square, using the image to be processed as the target image; and in response to determining that the image to be processed does not meet the requirements, adjusting the image to be processed to meet the requirements based on the principle of minimizing the impact on the image quality of the image to be processed, thereby obtaining the target image; The second processing module is used to respectively obtain the two-dimensional rotation position code corresponding to each image block, including: determining the target beta value corresponding to the target image, and performing the following processing for any image block: Beta encoding the x-direction position coordinates and the y-direction position coordinates of the image block according to the target beta value, using the obtained x-direction encoding results and the y-direction encoding results as the two-dimensional rotation position code of the image block, and using the visual base model to generate the image processing result corresponding to the target image based on each image block and the corresponding two-dimensional rotation position code.

7. The device according to claim 6, wherein In response to determining that the length and width of the target image are both less than or equal to a first value, the second processing module uses the training beta value used when training the visual base model as the target beta value, and the first value is the maximum value of the length and width of the training image used when training the visual base model.

8. The device according to claim 7, wherein The second processing module is further used to, in response to determining that at least one of the length and width of the target image is greater than the first value, determine the target beta value according to the training beta value in accordance with the principle that the x-direction encoding result and the y-direction encoding result corresponding to the target image are both within a predetermined value range, wherein the predetermined value range is the value range of the x-direction encoding result and the y-direction encoding result generated when training the visual base model.

9. The device according to claim 8, wherein The second processing module takes the larger value of the length and width of the target image as the second value, and calculates the target beta value through a predetermined transformation formula based on the second value, the first value and the training beta value.

10. A training device for a visual base model, comprising: Sample acquisition module and model training module; The sample acquisition module is used to acquire training samples, wherein each training sample includes: image blocks obtained by segmenting the training image and two-dimensional rotation position codes corresponding to each image block; wherein the two-dimensional rotation position codes corresponding to each image block include: x-direction encoding results and y-direction encoding results obtained by beta-encoding the x-direction position coordinates and the y-direction position coordinates of the image block according to the determined target beta value; The model training module is used to train the visual base model using the training samples, and the visual base model is used to generate an image processing result corresponding to the target image based on the image blocks obtained after segmenting the target image and the two-dimensional rotation position codes corresponding to the image blocks, so that there is no overlap between any two image blocks corresponding to the same image; The image block includes M pixels, M is a positive integer greater than one, and the image block is a square-shaped image block; The training image meets the following requirements: the length of the training image is an integer multiple of the side length of the square, and the width of the training image is an integer multiple of the side length of the square; The target image meets the following requirements: the length of the target image is an integer multiple of the side length of the square, and the width of the target image is an integer multiple of the side length of the square.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program / instructions, wherein when the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images