Image reading and writing method and device, electronic device and storage medium

By acquiring and calculating the storage address of the target sub-image, CPU operation is reduced, and the migration efficiency of feature maps in convolutional neural networks is improved, and the problem of frequent CPU operations during feature map migration between memory and external memory is solved.

CN112580485BActive Publication Date: 2024-11-12EDGELESS SEMICON CO LTD OF ZHUHAI +1
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202011475929.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-14
Publication Date
2024-11-12
Estimated Expiration
2040-12-14

AI Technical Summary

Technical Problem

Convolutional neural networks generate a large number of feature maps during the calculation process, resulting in memory limitations and frequent CPU operations, affecting the execution efficiency of neural network processors.

Method used

By obtaining the address parameters of the target subimage, using the address calculator to calculate and read the storage address of the target subimage, read and write it from external memory, reduce CPU operations and improve image migration efficiency.

Benefits of technology

Save CPU resources, improve the execution efficiency of image migration, and solve the problem of high CPU operation consumption during feature map migration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112580485B_ABST
    Figure CN112580485B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for reading and writing an image, a storage medium, and an electronic device, wherein the method includes: obtaining a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among multiple sub-images contained in the target image, and the first address parameter is used to determine the first external memory address of the target sub-image in the target external memory; calculating the first external memory address of the target sub-image in the target external memory from the external memory to the internal memory through an address calculator according to the first address parameter; reading the target sub-image from the target external memory according to the first external memory address, and writing the target sub-image to the first memory address in the target internal memory. Through the present application, a large amount of CPU operations do not need to be consumed, and the technical effect of saving CPU resources and improving the execution efficiency of image migration can be achieved, thereby solving the problem of high CPU operation consumption in the method of moving feature maps between internal memory and external memory in the related art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to an image reading and writing method and device, an electronic device and a storage medium. Background Art

[0002] The unique nonlinear adaptive information processing capability of convolutional neural networks overcomes the shortcomings of traditional artificial intelligence methods in image recognition, speech recognition, and unstructured information processing, making them successfully applied in pattern recognition, intelligent control, optimization combination, prediction, and other fields.

[0003] Convolutional neural networks contain multiple hidden layers, each of which generates multiple feature maps and outputs them to the next hidden layer.

[0004] During the calculation process of the convolutional neural network, its multi-layer network will generate a large number of feature maps. Due to space limitations of the memory, the feature maps need to be segmented and cropped, and the cropped images need to be spliced ​​in the external memory. This process will consume a lot of CPU operations, and the image movement speed will directly affect the execution efficiency of the neural network processor. Summary of the invention

[0005] The present application provides an image reading and writing method and device, a storage medium and an electronic device, so as to at least solve the problem in the related art that the moving speed of the feature image directly affects the execution efficiency of the neural network processor.

[0006] According to one aspect of an embodiment of the present application, a method for reading and writing an image is provided, the method comprising:

[0007] Acquire a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in the target external memory;

[0008] Calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by an address calculator;

[0009] The target sub-image is read from the target external memory according to the first external memory address, and the target sub-image is written to the first memory address in the target internal memory.

[0010] Optionally, calculating, by the address calculator according to the first address parameter, the first external memory address of the target sub-image in the target external memory comprises:

[0011] The first external memory address of the target sub-image in the target external memory is calculated by the address calculator according to the first starting address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image, wherein the first address parameters include: the first starting address, the first size, the second size, and the first address offset, the first starting address is the starting address of the target image stored in the target external memory, and the first address offset is the offset of the starting address of the target sub-image relative to the first starting address.

[0012] Optionally, calculating, by the address calculator according to the first address parameter, the first external memory address of the target sub-image in the target external memory comprises:

[0013] In the case that there are multiple target images and each of the target images reads the target sub-image at the same position, the first external memory address of the first target sub-image in the target external memory is calculated by the address calculator according to the first address parameter of the first target sub-image, wherein the multiple target images are stored continuously in the target external memory, and the first target sub-image corresponds to the first target image;

[0014] The first external memory address of the target sub-image of the other images is determined by the address calculator according to the first external memory address of the first target sub-image, the first size, and the number of images between the first target image and other images, wherein the other images are images among the multiple target images other than the first target image.

[0015] Optionally, before writing the target sub-image to the first memory address in the target memory, the method further includes:

[0016] Calculating the first memory address of the first target sub-image by the address calculator according to the second address parameter of the first target sub-image, wherein the second address parameter includes: the first address of the first target sub-image stored in the target memory, and the second size;

[0017] The first memory address of the target sub-image of the other images is determined by the address calculator according to the first memory address of the first target sub-image, the second size, and the number of images between the first target image and the other images, wherein the target sub-images of multiple target images are stored consecutively in sequence in the target memory.

[0018] Optionally, after writing the target sub-image to the first memory address in the target memory, the method further includes:

[0019] Obtaining a third address parameter of a plurality of target sub-feature maps, wherein the plurality of target sub-feature maps are feature maps obtained by performing convolution calculation on the target sub-image, and the third address parameter is used to determine a second memory address of the plurality of target sub-feature maps in the target memory;

[0020] Calculating, by the address calculator, the second memory addresses of the plurality of target sub-feature graphs in the target memory according to the third address parameter;

[0021] The plurality of target sub-feature maps are read from the target memory according to the second memory address, and the plurality of target sub-feature maps are written to the second external memory address in the target external memory.

[0022] Optionally, calculating, by the address calculator according to the third address parameter, the second memory addresses of the plurality of target sub-feature graphs in the target memory comprises:

[0023] Calculating the second memory address of the first target sub-feature map according to the second first address of the first target sub-feature map and the third size of the target sub-feature map by the address calculator, wherein the third address parameter includes the second first address and the third size, and the second first address is the first address of the first target sub-feature map stored in the target memory;

[0024] The second memory addresses of the other sub-feature graphs are determined by the address calculator according to the second memory address of the first target sub-feature graph, the third size, and the number of sub-feature graphs between the first target sub-feature graph and the other sub-feature graphs, wherein a plurality of the target sub-feature graphs are continuously stored in the target memory.

[0025] Optionally, before writing the plurality of target sub-feature maps to the second external memory address in the target external memory, the method further comprises:

[0026] The second external memory address of the first target sub-feature map is calculated by the address calculator according to the fourth address parameters of the plurality of target sub-feature maps, wherein the fourth address parameters include: the first address of the plurality of target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, the second address offset of the first target sub-feature map, the plurality of target feature maps are feature maps corresponding to the target image, the plurality of target feature maps are stored continuously in the target external memory, and the plurality of target feature maps are connected to the plurality of target sub-feature maps. Figure 1One-to-one correspondence, the first target sub-feature map corresponds to the first target feature map;

[0027] The second external memory address of the target sub-feature map of the other feature maps is determined by the address calculator according to the second external memory address of the first target sub-feature map, the fourth size, and the number of feature maps between the first target feature map and the other feature maps, wherein the target sub-images of multiple target images are stored continuously in sequence in the target memory.

[0028] According to another aspect of the embodiment of the present application, there is also provided an image reading and writing device, the device comprising:

[0029] An acquisition module, used for acquiring a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used for determining a first external memory address of the target sub-image in a target external memory;

[0030] A first calculation module, configured to calculate the first external memory address of the target sub-image in the target external memory according to the first address parameter by using an address calculator;

[0031] A reading module is used to read the target sub-image from the target external memory according to the first external memory address, and write the target sub-image to the first memory address in the target internal memory.

[0032] Optionally, the first calculation module includes:

[0033] A first calculation unit is used to calculate the first external memory address of the target sub-image in the target external memory through the address calculator according to the first starting address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image, wherein the first address parameters include: the first starting address, the first size, the second size, and the first address offset, the first starting address is the starting address of the target image stored in the target external memory, and the first address offset is the offset of the starting address of the target sub-image relative to the first starting address.

[0034] Optionally, the first computing unit includes:

[0035] a calculation subunit, configured to calculate the first external memory address of the first target sub-image in the target external memory by the address calculator according to the first address parameter of the first target sub-image, when there are multiple target images and each target image reads the target sub-image at the same position, wherein the multiple target images are stored continuously in the target external memory, and the first target sub-image corresponds to the first target image;

[0036] A determination subunit is used to determine the first external memory address of the target sub-image of the other images through the address calculator according to the first external memory address of the first target sub-image, the first size, and the number of images between the first target image and other images, wherein the other images are images other than the first target image among the multiple target images.

[0037] Optionally, the device further comprises:

[0038] A second calculation module is used for calculating the first memory address of the first target sub-image by the address calculator according to the second address parameter of the first target sub-image before writing the target sub-image into the first memory address in the target memory, wherein the second address parameter includes: the first address of the first target sub-image stored in the target memory, and the second size;

[0039] The first determination module is used to determine the first memory address of the target sub-image of the other images according to the first memory address of the first target sub-image, the second size, and the number of images between the first target image and the other images through the address calculator, wherein the target sub-images of multiple target images are stored continuously in sequence in the target memory.

[0040] Optionally, the device further comprises:

[0041] a second determination module, configured to obtain third address parameters of a plurality of target sub-feature maps after writing the target sub-image to the first memory address in the target memory, wherein the plurality of target sub-feature maps are feature maps obtained by performing convolution calculation on the target sub-image, and the third address parameters are used to determine a second memory address of the plurality of target sub-feature maps in the target memory;

[0042] A third calculation module, configured to calculate the second memory addresses of the plurality of target sub-feature graphs in the target memory according to the third address parameter by means of the address calculator;

[0043] A writing module is used to read the multiple target sub-feature maps from the target memory according to the second memory address, and write the multiple target sub-feature maps to the second external memory address in the target external memory.

[0044] Optionally, the third calculation module includes:

[0045] a second calculation unit, configured to calculate the second memory address of the first target sub-feature map according to the second first address of the first target sub-feature map and the third size of the target sub-feature map through the address calculator, wherein the third address parameter includes the second first address and the third size, and the second first address is the first address of the first target sub-feature map stored in the target memory;

[0046] a determining unit, configured to determine the second memory addresses of the other sub-feature graphs by the address calculator according to the second memory address of the first target sub-feature graph, the third size, and the number of sub-feature graphs spaced between the first target sub-feature graph and the other sub-feature graphs, wherein the plurality of target sub-feature graphs are continuously stored in the target memory.

[0047] Optionally, the device further comprises:

[0048] a fourth calculation module, used for calculating the second external memory address of the first target sub-feature map by the address calculator according to the fourth address parameters of the plurality of target sub-feature maps before writing the plurality of target sub-feature maps to the second external memory address in the target external memory, wherein the fourth address parameters include: the first address of the plurality of target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, the second address offset of the first target sub-feature map, the plurality of target feature maps being feature maps corresponding to the target image, the plurality of target feature maps being continuously stored in the target external memory, and the plurality of target feature maps being consecutively stored in the target external memory. Figure 1 One-to-one correspondence, the first target sub-feature map corresponds to the first target feature map;

[0049] The third determination module is used to determine the second external memory address of the target sub-feature map of the other feature maps according to the second external memory address of the first target sub-feature map, the fourth size, and the number of feature maps between the first target feature map and the other feature maps through the address calculator, wherein the target sub-images of multiple target images are stored continuously in sequence in the target memory.

[0050] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein the memory is used to store a computer program; and the processor is used to execute the steps of the image reading and writing method in any of the above embodiments by running the computer program stored in the memory.

[0051] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of the image reading and writing method in any of the above embodiments when run.

[0052] According to another aspect of the embodiments of the present application, a computer program product or a computer program is also provided, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the image reading and writing method in any of the above embodiments.

[0053] In an embodiment of the present application, based on the address at which the sub-image to be convolved among the multiple sub-images contained in the target image is stored in the target external memory, the target sub-image to be convolved is obtained, and after the target sub-image is read out from the target external memory, it is written to the memory address of the target internal memory. Since the image storage address is calculated according to the address parameters by the address calculator, and the image is read and written according to the image storage address, a large number of CPU operations do not need to be consumed, and the technical effect of saving CPU resources and improving the execution efficiency of image migration can be achieved, thereby solving the problem of high CPU operation consumption in the method of moving feature maps between internal memory and external memory in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 is a flowchart of an optional method for reading and writing an image according to an embodiment of the present application;

[0057] Figure 2 is an optional characteristic graph reading and writing circuit block diagram according to an embodiment of the present application;

[0058] Figure 3 is a schematic diagram of an optional feature map reading process according to an embodiment of the present application;

[0059] Figure 4 It is a schematic diagram of an optional feature map writing process according to an embodiment of the present application;

[0060] Figure 5 is a structural block diagram of an optional image reading and writing device according to an embodiment of the present application;

[0061] Figure 6 It is a structural block diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0063] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] As a main branch of deep neural networks, the convolutional neural network has a network structure consisting of an input layer, an output layer, and a multi-level hidden layer. During the convolutional neural network calculation process, each layer will generate a large number of feature maps. Due to capacity limitations, the memory cannot store all feature maps, so the feature maps need to be transferred between the memory and the external memory. This transfer process consumes a lot of CPU operations. In order to solve this problem, the embodiment of the present application provides a method for reading and writing images, such as Figure 1 As shown, the method includes:

[0065] Step S101, obtaining a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in the target external memory;

[0066] Step S102, calculating a first external memory address of a target sub-image in a target external memory according to a first address parameter by an address calculator;

[0067] Step S103, reading the target sub-image from the target external memory according to the first external memory address, and writing the target sub-image to the first memory address in the target internal memory.

[0068] Optionally, the embodiment of the present application first needs to acquire at least one target image, which may be an image of three RGB channels without convolution operation, or a feature map obtained after convolution.

[0069] Then, the sub-image to be convolved among the multiple sub-images contained in the target image is taken as the target sub-image, and the first address parameter of the target sub-image is obtained. After the first address parameter is input into the address calculator, the address of the target sub-image in the target external memory (i.e., the first external memory address) can be obtained, and the target sub-image is read out from the target external memory using the first external memory address of the target sub-image, and written to the first memory address in the target internal memory. It should be noted that the first memory address is the address for storing the target sub-image.

[0070] In an embodiment of the present application, based on the address at which the sub-image to be convolved among the multiple sub-images contained in the target image is stored in the target external memory, the target sub-image to be convolved is obtained, and after the target sub-image is read out from the target external memory, it is written to the memory address of the target internal memory. Since the image storage address is calculated according to the address parameters by the address calculator, and the image is read and written according to the image storage address, a large number of CPU operations do not need to be consumed, and the technical effect of saving CPU resources and improving the execution efficiency of image migration can be achieved, thereby solving the problem of high CPU operation consumption in the method of moving feature maps between internal memory and external memory in the related art.

[0071] As an optional embodiment, calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by the address calculator includes:

[0072] The first external memory address of the target sub-image in the target external memory is calculated by an address calculator according to the first starting address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image, wherein the first address parameters include: the first starting address, the first size, the second size, and the first address offset, the first starting address is the starting address of the target image stored in the target external memory, and the first address offset is the offset of the starting address of the target sub-image relative to the first starting address.

[0073] Optionally, the first external memory address of the target sub-image output by the address calculator is obtained based on the first starting address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image, wherein the first address offset is the offset of the starting address of the target sub-image relative to the first starting address.

[0074] For example, after obtaining the first address of the target image stored in the target external memory, the offset of the first address of the target sub-image relative to the first first address, the first size of the target image and the second size of the target sub-image, it is possible to locate the address of the target sub-image stored in the target external memory.

[0075] As an optional embodiment, calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by the address calculator includes:

[0076] In the case where there are multiple target images and each target image reads a target sub-image at the same position, the first external memory address of the first target sub-image in the target external memory is calculated by an address calculator according to a first address parameter of the first target sub-image, wherein the multiple target images are continuously stored in the target external memory, and the first target sub-image corresponds to the first target image;

[0077] The first external memory addresses of target sub-images of other images are determined by an address calculator according to the first external memory address, the first size, and the number of images between the first target image and other images, wherein the other images are images other than the first target image among the multiple target images.

[0078] Optionally, when there are multiple target images and the multiple target images are stored continuously in the target external memory, according to the way in which the convolutional neural network performs convolution processing on the image, it is necessary to obtain the target sub-image at the same position of each target image. In order to obtain the target sub-image at the same position of each target image, the storage address (i.e., the first external memory address) of the target sub-image in each target image in the target external memory can be determined by obtaining the first external memory address of the first target sub-image in the target external memory, the first size of the target image, and the number of images separated from the first target image by other images.

[0079] For example, the first external memory address of the first target sub-image in the target external memory is a, and the first size of the target image is b, where b = width * height of the target image. At this time, the number of images between the second target image and the first target image is 0, and the first external memory address of the target sub-image of the second target image in the target external memory is a+b. The number of images between the third target image and the first target image is 1, and the first external memory address of the target sub-image of the third target image in the target external memory is a+b+b. By analogy, the first external memory addresses of the target sub-images of other images can be obtained.

[0080] As an optional embodiment, before writing the target sub-image to the first memory address in the target memory, the method further includes:

[0081] Calculating a first memory address of the first target sub-image according to a second address parameter of the first target sub-image by an address calculator, wherein the second address parameter includes: a first address of the first target sub-image stored in the target memory and a second size;

[0082] The first memory addresses of target sub-images of other images are determined by an address calculator according to the first memory address, the second size, and the number of images between the first target image and other images, wherein the target sub-images of multiple target images are stored consecutively in the target memory in sequence.

[0083] Optionally, when target sub-images of multiple target images are stored continuously in the target memory, the storage address (i.e., the first memory address) of the target sub-image in each target image in the target memory can be determined by obtaining the first memory address of the first target sub-image in the target memory, the second size of the target sub-image, and the number of images between the other images and the first target image.

[0084] For example, the first memory address of the first target sub-image in the target memory is m, and the second size of the target sub-image is n, where n = width * height of the target sub-image. At this time, the number of images between the second target image and the first target image is 0, and the first memory address of the target sub-image of the second target image in the target memory is m+n. The number of images between the third target image and the first target image is 1, and the first memory address of the target sub-image of the third target image in the target memory is m+n+n. By analogy, the first memory addresses of the target sub-images of other images can be obtained.

[0085] As an optional embodiment, the method further includes:

[0086] After writing the target sub-image to the first memory address in the target memory, obtaining third address parameters of the plurality of target sub-feature maps, wherein the plurality of target sub-feature maps are feature maps obtained by performing convolution calculation on the target sub-image, and the third address parameters are used to determine the second memory addresses of the plurality of target sub-feature maps in the target memory;

[0087] Calculating, by an address calculator, second memory addresses of the plurality of target sub-feature graphs in the target memory according to a third address parameter;

[0088] A plurality of target sub-feature maps are read from the target memory according to the second memory address, and the plurality of target sub-feature maps are written to the second external memory address in the target external memory.

[0089] Optionally, a convolutional neural network is used to perform convolution calculation on the target sub-image to obtain third address parameters of multiple target sub-feature maps.

[0090] The address calculator calculates the third address parameter to determine the second memory address of the multiple target sub-feature maps obtained after convolution in the target memory.

[0091] Then, according to the second memory address, multiple target sub-feature graphs are read from the target memory and written to the second external memory address in the target external memory, wherein the second external memory address is used to store multiple target sub-feature graphs.

[0092] As an optional embodiment, calculating the second memory addresses of the plurality of target sub-feature graphs in the target memory according to the third address parameter by the address calculator includes:

[0093] Calculating a second memory address of the first target sub-feature map according to the second first address of the first target sub-feature map and the third size of the target sub-feature map by an address calculator, wherein the third address parameter includes the second first address and the third size, and the second first address is the first address of the first target sub-feature map stored in the target memory;

[0094] The second memory addresses of other sub-feature graphs are determined by an address calculator according to the second memory address of the first target sub-feature graph, the third size, and the number of sub-feature graphs between the first target sub-feature graph and the other sub-feature graphs, wherein the multiple target sub-feature graphs are continuously stored in the target memory.

[0095] Optionally, the second address of the first target sub-feature map is first obtained, and then the third size of the target sub-feature map is obtained. Since the multiple target sub-feature maps in the embodiment of the present application are stored continuously in the target memory, the address calculator can determine the second memory addresses of other sub-feature maps in the target memory based on the second address of the first target sub-feature map, the third size of the target sub-feature map, and the number of sub-feature maps between the first target sub-feature map and the other sub-feature maps.

[0096] For example, the second address of the first target sub-feature map is p, and the third size of the target sub-feature map is q, where q = width * height of the target sub-feature map. At this time, the number of images between the second target sub-feature map and the first target sub-feature map is 0, and the second memory address of the second target sub-feature map is p+q. The number of images between the third target sub-feature map and the first target sub-feature map is 1, and the second memory address of the third target sub-feature map is p+q+q. By analogy, the second memory addresses of other target sub-feature maps can be obtained.

[0097] As an optional embodiment, the method further includes:

[0098] Before writing the plurality of target sub-feature maps to the second external memory address in the target external memory, the second external memory address of the first target sub-feature map is calculated by an address calculator according to the fourth address parameters of the plurality of target sub-feature maps, wherein the fourth address parameters include: the first address of the plurality of target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, the second address offset of the first target sub-feature map, the plurality of target feature maps are feature maps corresponding to the target image, the plurality of target feature maps are continuously stored in the target external memory, and the plurality of target feature maps are connected to the plurality of target sub-feature maps. Figure 1 One-to-one correspondence, the first target sub-feature map corresponds to the first target feature map;

[0099] The second external memory addresses of target sub-feature maps of other feature maps are determined by an address calculator according to the second external memory address of the first target sub-feature map, the fourth size, and the number of feature maps between the first target feature map and the other feature maps, wherein the target sub-images of multiple target images are stored consecutively in the target memory in sequence.

[0100] Optionally, according to the first size of the target image and the convolution kernel size, multiple target feature maps can be obtained in advance to obtain a fourth size of the target feature map.

[0101] According to the first address of the multiple target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, and the second address offset of the first target sub-feature map, the fourth address parameters of the multiple target sub-feature maps are obtained, wherein the multiple target feature maps are continuously stored in the target external memory, and the multiple target feature maps and the multiple target sub-feature maps are connected to each other. Figure 1 Correspondingly, the second address offset is the offset of the first target sub-feature map relative to the first target feature map.

[0102] The address calculator determines the second external memory address of the target sub-feature map of other feature maps according to the second external memory address of the first target sub-feature map, the fourth size of the target feature map and the number of feature maps between the first target feature map and other feature maps.

[0103] For example, the second external memory address of the first target sub-feature map of the first target feature map is g, and the fourth dimension of the target feature map is h, where h = width * height of the target feature map. At this time, the number of feature maps between the second target feature map and the first target feature map is 0, and the second external memory address of the first target sub-feature map of the second target feature map is g+h. The number of feature maps between the third target feature map and the first target feature map is 1, and the second external memory address of the first target sub-feature map of the third target feature map is g+h+h. By analogy, the second memory address of the first target sub-feature map in other feature maps can be obtained.

[0104] The second external memory address of the second target sub-feature map of the first target feature map is j, and the fourth size of the target feature map is h, where h = width * height of the target feature map. At this time, the number of feature maps between the second target feature map and the first target feature map is 0, and the second external memory address of the second target sub-feature map of the second target feature map is j+h. The number of feature maps between the third target feature map and the first target feature map is 1, and the second external memory address of the second target sub-feature map of the third target feature map is j+h+h. By analogy, the second memory address of the second target sub-feature map in other feature maps can be obtained.

[0105] Similarly, the second external memory addresses of other target sub-feature maps of other target feature maps are also calculated according to the above method, which will not be repeated here.

[0106] In the target external memory, each target feature map is concatenated using the second external memory addresses of all target sub-feature maps to represent the image feature information of the target image.

[0107] In addition, the embodiment of the present application sets the image bit width to represent the number of pixel bits occupied by the image. When obtaining the size of the target feature map, target sub-image, and target sub-feature map, the image bit width can be multiplied to represent the number of pixel bits occupied by each target feature map, target sub-image, and target sub-feature map.

[0108] The embodiment of the present application can derive the storage location of each target sub-feature map in each target feature map in the target external memory based on the fourth address parameters of the multiple target sub-feature maps obtained by calculation, and automatically complete the automatic movement of the target feature map between the target internal memory and the target external memory without the need for CPU operation, thereby releasing CPU resources.

[0109] As an optional embodiment, the present application embodiment provides a system for reading and writing an image, such as Figure 2 ,3,4, it can be seen that the system includes: external memory, internal memory, parameter configuration module, address calculation module, bus control interface module, data cache module, feature map reading module, feature map writing module;

[0110] The external memory is connected to the bus control interface module, the data cache module is connected to the bus control interface module, the feature map writing module and the feature map reading module, the parameter configuration module is connected to the address calculation module, the feature map reading module and the feature map writing module are connected to the address calculation module, the bus control interface module and the internal memory respectively;

[0111] The parameter configuration module is used to set parameters, such as the size and number of input feature maps, the size and offset address of input original images, or the size and number of output feature maps, the size and offset address of output original images;

[0112] An address calculation module is used to calculate the address of the input / output feature map in the memory / external storage according to the parameter configuration and pass it to the feature map reading / writing module;

[0113] The feature map reading module is used to read the feature map input from the external memory from the data cache module, input the address of the feature map and the first address of the memory into the address calculation module, and write the feature map to the corresponding address in the memory;

[0114] The feature map writing module is used to write the feature map written out of the memory into the data cache module, output the address of the feature map in the address calculation module, and write the feature map to the corresponding address in the external memory through the bus control interface module;

[0115] The bus control interface module is used to generate bus read and write signals according to the address information transmitted by the feature map reading module and the feature map writing module and the data cache module status, and perform read and write operations on the external memory.

[0116] It can be understood that in the embodiment of the present application, the external memory is equal to the target external memory, and the internal memory is equal to the target memory.

[0117] In addition, the feature map reading module sends a read request to the bus control interface module according to the external memory address and the image location. For example, when the read image is located at the beginning or middle position of a row of the feature map, because the subsequent images are placed continuously in the external memory, a continuous read request (such as continuous reading of 4 or 8 images) can be sent to improve the reading efficiency. For example, when the read image is located at the end of a row of the feature map (such as Figure 3 Point P as shown), since the subsequent image address needs to jump to the starting position of the next row, a single read request is required at this time. When the feature map reading module issues a read request, if there is data update in the data cache module, the data of the cache module is read and written into the memory according to the memory address; the bus control interface module issues a bus read request according to the read request and address of the feature map reading module and the status of the data cache module, and stores the read data returned by the bus in the data cache module. If the status of the data cache module is full, it waits until the status of the data cache module can store the data of the read request, and then a read request is issued.

[0118] When executing the feature map write function, such as Figure 4 The parameter configuration is the same as above, except that it is changed to output feature maps and output original images. The address calculation method is the same. The feature map write module sends a write request (continuous write request or single write request) to the bus module according to the address and image location, and writes the output feature map in the memory into the data cache module. The bus control module sends a bus write request based on the write request, address and status of the data cache module, and reads the data from the data cache module and sends it to the bus. Similarly, when the data cache module does not have enough data for the write request, it waits until the data cache module receives enough data before sending a write request.

[0119] When realizing efficient reading and writing of multiple feature maps, it is only necessary to change the first address of the current feature map being read or written to complete the reading and splicing of any feature map.

[0120] According to another aspect of an embodiment of the present application, a device for reading and writing an image for implementing the above-mentioned method for reading and writing an image is also provided. Figure 5 is a schematic diagram of an optional image reading and writing device according to an embodiment of the present application, such as Figure 5 As shown, the device may include:

[0121] An acquisition module 501 is used to acquire a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in a target external memory;

[0122] A first calculation module 502, used for calculating a first external memory address of a target sub-image in a target external memory according to a first address parameter by means of an address calculator;

[0123] The reading module 503 is used to read the target sub-image from the target external memory according to the first external memory address, and write the target sub-image to the first memory address in the target internal memory.

[0124] It should be noted that the acquisition module 501 in this embodiment can be used to execute the above step S101, the first calculation module 502 in this embodiment can be used to execute the above step S102, and the reading module 503 in this embodiment can be used to execute the above step S103.

[0125] As an optional embodiment, the first calculation module includes:

[0126] The first calculation unit is used to calculate the first external memory address of the target sub-image in the target external memory according to the first starting address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image through an address calculator, wherein the first address parameters include: the first starting address, the first size, the second size, and the first address offset, the first starting address is the starting address of the target image stored in the target external memory, and the first address offset is the offset of the starting address of the target sub-image relative to the first starting address.

[0127] As an optional embodiment, the first calculation unit includes:

[0128] a calculation subunit, for calculating a first external memory address of a first target sub-image in a target external memory by an address calculator according to a first address parameter of the first target sub-image when there are multiple target images and each target image reads a target sub-image at the same position, wherein the multiple target images are stored continuously in the target external memory and the first target sub-image corresponds to the first target image;

[0129] A determination subunit is used to determine the first external memory address of target sub-images of other images through an address calculator according to the first external memory address, the first size, and the number of images between the first target image and other images, wherein the other images are images other than the first target image among the multiple target images.

[0130] As an optional embodiment, the device further includes:

[0131] a second calculation module, used for calculating the first memory address of the first target sub-image by an address calculator according to the second address parameter of the first target sub-image before writing the target sub-image to the first memory address in the target memory, wherein the second address parameter includes: the first address of the first target sub-image stored in the target memory, and the second size;

[0132] The first determination module is used to determine the first memory address of the target sub-image of other images according to the first memory address, the second size, and the number of images between the first target image and other images through an address calculator, wherein the target sub-images of multiple target images are stored continuously in the target memory in sequence.

[0133] As an optional embodiment, the device further includes:

[0134] a second determination module, configured to obtain third address parameters of a plurality of target sub-feature maps after writing the target sub-image to the first memory address in the target memory, wherein the plurality of target sub-feature maps are feature maps obtained by performing convolution calculation on the target sub-image, and the third address parameters are used to determine a second memory address of the plurality of target sub-feature maps in the target memory;

[0135] A third calculation module, used for calculating second memory addresses of the plurality of target sub-feature graphs in the target memory according to a third address parameter by means of an address calculator;

[0136] The writing module is used to read multiple target sub-feature maps from the target memory according to the second memory address, and write the multiple target sub-feature maps to the second external memory address in the target external memory.

[0137] As an optional embodiment, the third calculation module includes:

[0138] a second calculation unit, configured to calculate a second memory address of the first target sub-feature map according to a second first address of the first target sub-feature map and a third size of the target sub-feature map through an address calculator, wherein the third address parameter includes the second first address and the third size, and the second first address is a first address of the first target sub-feature map stored in the target memory;

[0139] The determining unit is used to determine the second memory addresses of other sub-feature graphs according to the second memory address of the first target sub-feature graph, the third size, and the number of sub-feature graphs separated from the first target sub-feature graph and the other sub-feature graphs through an address calculator, wherein the multiple target sub-feature graphs are continuously stored in the target memory.

[0140] As an optional embodiment, the device further includes:

[0141] a fourth calculation module, used for calculating the second external memory address of the first target sub-feature map by using an address calculator according to fourth address parameters of the plurality of target sub-feature maps before writing the plurality of target sub-feature maps to the second external memory address in the target external memory, wherein the fourth address parameters include: the first address of the plurality of target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, the second address offset of the first target sub-feature map, the plurality of target feature maps being feature maps corresponding to the target image, the plurality of target feature maps being continuously stored in the target external memory, and the plurality of target feature maps being directly connected to the plurality of target sub-feature maps; Figure 1 One-to-one correspondence, the first target sub-feature map corresponds to the first target feature map;

[0142] The third determination module is used to determine the second external memory address of the target sub-feature map of other feature maps according to the second external memory address of the first target sub-feature map, the fourth size, and the number of feature maps between the first target feature map and the other feature maps through an address calculator, wherein the target sub-images of multiple target images are stored continuously in the target memory in sequence.

[0143] According to another aspect of an embodiment of the present application, an electronic device for implementing the above-mentioned method for controlling food in a refrigerator is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0144] Figure 6 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 communicate with each other through the communication bus 604, wherein,

[0145] Memory 603, used for storing computer programs;

[0146] The processor 601 is used to implement the following steps when executing the computer program stored in the memory 603:

[0147] S1, obtaining a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in the target external memory;

[0148] S2, calculating a first external memory address of the target sub-image in the target external memory according to the first address parameter by an address calculator;

[0149] S3, reading the target sub-image from the target external memory according to the first external memory address, and writing the target sub-image to the first memory address in the target internal memory.

[0150] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0151] The communication interface is used for communication between the above electronic device and other devices.

[0152] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0153] As an example, Figure 6 As shown, the memory 603 may include but is not limited to the acquisition module 501, the first calculation module 502, and the reading module 503 in the image reading and writing device. In addition, it may also include but is not limited to other module units in the image reading and writing device, which will not be repeated in this example.

[0154] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0155] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0156] It can be understood by those skilled in the art that Figure 6The structure shown is for illustration only. The device for implementing the above-mentioned image reading and writing method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 6 It does not limit the structure of the above electronic device. For example, the terminal device may also include Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 6 Different configurations shown.

[0157] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0158] According to another aspect of the embodiment of the present application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of the image reading and writing method.

[0159] Optionally, in this embodiment, the storage medium may be located on at least one network device among a plurality of network devices in the network shown in the above embodiment.

[0160] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0161] S1, obtaining a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in the target external memory;

[0162] S2, calculating a first external memory address of the target sub-image in the target external memory according to the first address parameter by an address calculator;

[0163] S3, reading the target sub-image from the target external memory according to the first external memory address, and writing the target sub-image to the first memory address in the target internal memory.

[0164] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, which will not be described in detail in this embodiment.

[0165] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0166] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers or network devices, etc.) to execute all or part of the steps of the image reading and writing methods of each embodiment of the present application.

[0167] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0169] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for reading and writing an image, characterized in that: The method comprises: Acquire a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in the target image, and the first address parameter is used to determine a first external memory address of the target sub-image in a target external memory, wherein the first address parameter includes: a first starting address, a first size of the target image, a second size of the target sub-image, and a first address offset, wherein the first starting address is a starting address of the target image stored in the target external memory, and the first address offset is an offset of the starting address of the target sub-image relative to the first starting address; Calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by an address calculator; Reading the target sub-image from the target external memory according to the first external memory address, and writing the target sub-image to the first memory address in the target internal memory; Obtaining third address parameters of a plurality of target sub-feature maps, wherein the plurality of target sub-feature maps are feature maps obtained by performing convolution calculation on the target sub-image, and the third address parameters are used to determine second memory addresses of the plurality of target sub-feature maps in the target memory, wherein the third address parameters include a second first address and a third size, and the second first address is a first address of the first target sub-feature map stored in the target memory; Calculating, by the address calculator, the second memory addresses of the plurality of target sub-feature graphs in the target memory according to the third address parameter; The plurality of target sub-feature maps are read from the target memory according to the second memory address, and the plurality of target sub-feature maps are written to the second external memory address in the target external memory.

2. The method according to claim 1, characterized in that Calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by the address calculator includes: The first external memory address of the target sub-image in the target external memory is calculated by the address calculator according to the first head address of the target image, the first size of the target image, the second size of the target sub-image, and the first address offset of the target sub-image.

3. The method according to claim 2, characterized in that Calculating the first external memory address of the target sub-image in the target external memory according to the first address parameter by the address calculator includes: In the case that there are multiple target images and each of the target images reads the target sub-image at the same position, the first external memory address of the first target sub-image in the target external memory is calculated by the address calculator according to the first address parameter of the first target sub-image, wherein the multiple target images are stored continuously in the target external memory, and the first target sub-image corresponds to the first target image; The first external memory address of the target sub-image of the other images is determined by the address calculator according to the first external memory address of the first target sub-image, the first size, and the number of images between the first target image and other images, wherein the other images are images among the multiple target images other than the first target image.

4. The method according to claim 3, characterized in that Before writing the target sub-image to the first memory address in the target memory, the method further includes: Calculating the first memory address of the first target sub-image according to the second address parameter of the first target sub-image by the address calculator, wherein the second address parameter includes: the first address of the first target sub-image stored in the target memory and the second size; The first memory address of the target sub-image of the other images is determined by the address calculator according to the first memory address of the first target sub-image, the second size, and the number of images between the first target image and the other images, wherein the target sub-images of multiple target images are stored consecutively in sequence in the target memory.

5. The method according to claim 1, characterized in that Calculating, by the address calculator according to the third address parameter, the second memory addresses of the plurality of target sub-feature graphs in the target memory comprises: Calculating the second memory address of the first target sub-feature map by the address calculator according to the second first address of the first target sub-feature map and the third size of the target sub-feature map; The second memory addresses of the other sub-feature graphs are determined by the address calculator according to the second memory address of the first target sub-feature graph, the third size, and the number of sub-feature graphs between the first target sub-feature graph and the other sub-feature graphs, wherein a plurality of the target sub-feature graphs are continuously stored in the target memory.

6. The method according to claim 1, characterized in that Before writing the plurality of target sub-feature maps to the second external memory address in the target external memory, the method further comprises: Calculating the second external memory address of the first target sub-feature map by the address calculator according to the fourth address parameters of the plurality of target sub-feature maps, wherein the fourth address parameters include: the first address of the plurality of target feature maps in the target external memory, the fourth size of the target feature map, the third size of the target sub-feature map, the second address offset of the first target sub-feature map, the plurality of target feature maps are feature maps corresponding to the target image, the plurality of target feature maps are continuously stored in the target external memory, the plurality of target feature maps correspond one-to-one to the plurality of target sub-feature maps, and the first target sub-feature map corresponds to the first target feature map; The second external memory address of the target sub-feature map of the other feature maps is determined by the address calculator according to the second external memory address of the first target sub-feature map, the fourth size, and the number of feature maps between the first target feature map and the other feature maps, wherein the target sub-images of multiple target images are stored continuously in sequence in the target memory.

7. An image reading and writing device, characterized in that: The device comprises: an acquisition module, used for acquiring a first address parameter of a target sub-image, wherein the target sub-image is a sub-image to be convolved among a plurality of sub-images contained in a target image, and the first address parameter is used for determining a first external memory address of the target sub-image in a target external memory, wherein the first address parameter includes: a first starting address, a first size of the target image, a second size of the target sub-image, and a first address offset, wherein the first starting address is a starting address of the target image stored in the target external memory, and the first address offset is an offset of the starting address of the target sub-image relative to the first starting address; A calculation module, configured to calculate the first external memory address of the target sub-image in the target external memory according to the first address parameter by using an address calculator; A reading module is used to read the target sub-image from the target external memory according to the first external memory address, and write the target sub-image to the first memory address in the target internal memory.

8. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein the program, when executed, executes the steps of the image reading and writing method described in any one of claims 1 to 6.

9. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface, and the memory communicate with each other via a communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the steps of the image reading and writing method according to any one of claims 1 to 6 by running a program stored in a memory.

Citation Information

Patent Citations

  • Painting style conversion method and related products

    CN109117948A

  • Information processing method and terminal equipment

    CN109726806A

  • GPU sub-image processing method based on DMA

    CN111080508A

  • Artificial neural network processing system and data processing method thereof

    CN111160545A

  • Neural network computing device, data reading method, data storage method and related equipment

    CN111984189A