Image information extraction method, related device, system and storage medium

Abstract

This application discloses an image information extraction method, related apparatus, system, and storage medium. The image information extraction method includes: a cloud device extracting information from an image to be extracted sent by a terminal device to obtain a target image containing target information; determining mapping parameters based on the image information in the image to be extracted and the target image, the mapping parameters representing the mapping relationship between the target image and the image to be extracted; and sending the mapping parameters to the terminal device, which can then process the image to be extracted based on the mapping parameters to obtain the target image. The amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image. This solution can reduce the amount of data transmitted between the cloud device and the terminal device.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image information extraction method, related apparatus, system, and storage medium. Background Technology

[0002] With the development of science and technology, image processing technology is being applied more and more widely. However, the current image processing algorithms have relatively high computational costs, making them unsuitable for use on terminal devices. As a result, the image processing technologies used on terminal devices are relatively simple and can only perform basic image processing, leading to inaccurate image processing results. Summary of the Invention

[0003] This application provides at least one image information extraction method, related apparatus, system, and storage medium.

[0004] This application provides an image information extraction method, comprising: a cloud device extracting information from an image to be extracted sent by a terminal device to obtain a target image containing target information; determining mapping parameters based on the image information in the image to be extracted and the target image, wherein the mapping parameters are used to represent the mapping relationship between the target image and the image to be extracted; and sending the mapping parameters to the terminal device, wherein the terminal device can process the image to be extracted based on the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0005] This application provides an image information extraction method, comprising: a terminal device sending an image to be extracted to a cloud device; receiving mapping parameters sent by the cloud device, the mapping parameters representing the mapping relationship between a target image and the image to be extracted, the target image being obtained by the cloud device from the image to be extracted and containing target information; and processing the image to be extracted using the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0006] This application provides an image information extraction method, comprising: a terminal device sending an image to be extracted to a cloud device; the cloud device extracting information from the image to be extracted sent by the terminal device to obtain a target image containing target information; the cloud device determining mapping parameters based on the image information in the image to be extracted and the target image, the mapping parameters representing the mapping relationship between the target image and the image to be extracted; the cloud device sending the mapping parameters to the terminal device; and the terminal device processing the image to be extracted using the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0007] This application provides a first image information extraction device, comprising: an information extraction module, used by a cloud device to extract information from an image to be extracted sent by a terminal device to obtain a target image containing target information; a mapping parameter determination module, used to determine mapping parameters based on image information in the image to be extracted and the target image, wherein the mapping parameters represent the mapping relationship between the target image and the image to be extracted; and a communication module, used to send the mapping parameters to the terminal device, wherein the terminal device can process the image to be extracted based on the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0008] This application provides a second image information extraction device, comprising: an information sending module for a terminal device to send an image to be extracted to a cloud device; an information receiving module for receiving mapping parameters sent by the cloud device, the mapping parameters representing the mapping relationship between a target image and the image to be extracted, the target image being obtained by the cloud device through information extraction of the image to be extracted and containing target information; and a processing module for processing the image to be extracted using the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0009] This application provides a cloud device, including a first memory and a first processor, wherein the first processor is used to execute program instructions stored in the first memory to implement the first image information extraction method described above.

[0010] This application provides a terminal device, including a second memory and a second processor. The second processor is used to execute program instructions stored in the second memory to implement the second image information extraction method described above.

[0011] This application provides a terminal device, and the system includes the aforementioned cloud device and the aforementioned terminal device.

[0012] This application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement any of the above-described image information extraction methods.

[0013] In the above scheme, after receiving the image to be extracted from the terminal device, the cloud device extracts information from the image to obtain a target image containing the target information. Then, based on the image information between the image to be extracted and the target image, it determines the mapping parameters between them and sends these mapping parameters to the terminal device. This allows the terminal device to process the image to be extracted based on these mapping parameters to obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device side, enabling the use of more powerful algorithms to extract information from the image to be extracted, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image, thus improving the transmission efficiency between the cloud device and the terminal device.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0016] Figure 1 This is a flowchart illustrating an embodiment of the image information extraction method of this application;

[0017] Figure 2 This is a schematic diagram illustrating bilinear interpolation in one embodiment of the image information extraction method of this application;

[0018] Figure 3 This is a schematic diagram of a sub-process of step S12 in an embodiment of the image information extraction method of this application;

[0019] Figure 4 This is a schematic diagram showing the image block division result of an embodiment of the image information extraction method of this application;

[0020] Figure 5 This is a flowchart illustrating another embodiment of the image information extraction method of this application;

[0021] Figure 6 This is a schematic diagram illustrating the interaction between a cloud device and a terminal device, representing another embodiment of the image information extraction method of this application.

[0022] Figure 7 This is a schematic diagram of the structure of an embodiment of the first image information extraction device of this application;

[0023] Figure 8 This is a schematic diagram of the structure of an embodiment of the second image information extraction device of this application;

[0024] Figure 9 This is a schematic diagram of the structure of an embodiment of the cloud device of this application;

[0025] Figure 10 This is a schematic diagram of the structure of an embodiment of the terminal device of this application;

[0026] Figure 11 This is a schematic diagram of the structure of an embodiment of the image information extraction system of this application;

[0027] Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0028] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0030] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] Please see Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the image information extraction method of this application. Figure 1 As shown, the image information extraction method provided in this embodiment may include the following steps:

[0032] Step S11: The cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing target information.

[0033] The execution entity in this embodiment is a cloud device. The image to be extracted can be any image that requires information extraction. The image to be extracted can be a two-dimensional image or a three-dimensional image. Information extraction can be any method, such as high-frequency information extraction, low-frequency information extraction, feature extraction, foreground extraction, edge extraction, etc. For example, if it is necessary to extract a target image containing low-frequency information, the information extraction method is low-frequency information extraction; if it is necessary to extract a target image containing high-frequency information, the information extraction method is high-frequency information extraction; if it is necessary to extract a target image containing foreground information, the information extraction method is foreground extraction; if it is necessary to extract a target image containing edge information, the information extraction method is edge extraction. The target information and the method of target information extraction will not be described in detail here. Because cloud devices have high computing power support, high-computing-power algorithms can be used for information extraction, making the extracted target image containing target information more accurate. This embodiment takes low-frequency information as the target information and low-frequency extraction as the information extraction method. The low-frequency image extracted can be used for image enhancement, image segmentation, image denoising, and image registration, among other subsequent processing.

[0034] Step S12: Determine the mapping parameters based on the image information in the image to be extracted and the target image.

[0035] Mapping parameters represent the mapping relationship between the target image and the image to be extracted. For example, the mapping relationship can specifically be the difference in pixel values ​​at the same location in the target image and the image to be extracted, or a conversion relationship of pixel values ​​at the same location. Since the target image is obtained by extracting information from the image to be extracted, it can be assumed that the information at the same location in the target image and the image to be extracted is correlated. Therefore, a mapping relationship between the two images can be established through this correlation. The amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image. Sending the mapping parameters to the terminal device in a smaller amount than sending the target image results in less time spent transmitting the mapping parameters compared to transmitting the target image, thus improving transmission efficiency.

[0036] Step S13: Send the mapping parameters to the terminal device.

[0037] By sending mapping parameters to the terminal device, the terminal device can process the image to be extracted based on the mapping parameters to obtain the target image. Specifically, the mapping parameters represent the mapping relationship between the image to be extracted and the target image, so the target image can be determined based on this mapping parameter and the image to be extracted.

[0038] In the above scheme, after receiving the image to be extracted from the terminal device, the cloud device extracts information from the image to obtain a target image containing the target information. Then, based on the image information between the image to be extracted and the target image, a mapping parameter between them is determined, and this mapping parameter is sent to the terminal device. This allows the terminal device to process the image to be extracted based on the mapping parameter to obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device side, allowing the use of more powerful algorithms to extract information from the image to be extracted, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to transmit the mapping parameter is less than the amount of data required to transmit the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0039] In some embodiments, the following steps may also be performed before performing step S11 above:

[0040] The system receives a downsampled image from a terminal device. This downsampled image is obtained by downsampling the image to be extracted from the terminal device. Then, the downsampled image is upsampled to obtain the image to be extracted. The upsampling method can be bilinear upsampling.

[0041] For a better understanding of bilinear upsampling, please refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating bilinear interpolation in one embodiment of the image information extraction method of this application. Figure 2 As shown, if the length and width of the downsampled image DI transmitted from the terminal device are cH and cW respectively, and the upsampling factor is ur, while ensuring that the size of the upsampled image is consistent with the size of the image to be extracted, i.e., H and W, then a pixel after upsampling, mapped to a pixel in the downsampled image DI by the upsampling factor ur, is: Assuming the coordinates E of the downsampled image DI are exactly mapped to point E, and its four nearest neighbors are A, B, C, and D, as shown in the figure below, the bilinear interpolation result Value for pixel E is then determined. E The formula can be found in formula (1):

[0042] Value E =BF×EJ×Value A +AF×EJ×Value B +DJ×EF×Value C +CJ×EF×Value D Formula (1);

[0043] Where BF, AF, EJ, DJ, CJ, and EF represent lengths. For example, depending on the properties of the image, the lengths of AB, AC, BD, and CD can all be 1. ValueA Value is the pixel value of pixel point A. B Value is the pixel value of pixel B. C Value is the pixel value of pixel point C. D Let be the pixel value of pixel D.

[0044] In some application scenarios, besides bilinear interpolation, there are also bicubic interpolation, bicubic interpolation, and direction-based interpolation algorithms. Generally, the upsampling algorithm of cloud devices needs to correspond to the downsampling method of terminal devices. If the terminal device uses bilinear downsampling, then the cloud device needs to use bilinear upsampling. Errors may exist during downsampling and long sampling, causing a slight difference between the upsampled image and the image before downsampling. However, this difference is small and negligible. Therefore, the mapping parameters between the upsampled image and the target image obtained in this embodiment can be used to represent the mapping relationship between the image before downsampling and the target image.

[0045] In some embodiments, step S11 may include the following steps: using the image to be extracted as a guide map, performing guided filtering on the image to be extracted to obtain an image containing low-frequency information. The image containing low-frequency information is the target image. In some application scenarios, cloud devices use guided filtering to extract low-frequency information from the image to be extracted. Furthermore, guided filtering requires a guide map; in this embodiment, the image to be extracted is directly used as the guide map for guided filtering. The specific method of guided filtering can be found in formulas (2) and (3).

[0046]

[0047] Among them, clowUI i This refers to the target information at coordinate point i in the target image obtained after guided filtering, such as the pixel value of coordinate point i. represents the variance of a neighborhood of coordinate point i in the guided image. The neighborhood generally refers to the area within a preset range centered at coordinate point i. For example, a neighborhood centered at coordinate point i with a length of two pixels and a width of two pixels is used. This method of determining the neighborhood is merely an example; in other embodiments, a preset radius or diameter can also be used to determine the size of the neighborhood. ∈ is an external configuration parameter that adjusts the strength of the guided filter. Generally, the larger ∈ is, the stronger the guided filter. UI represents the image to be extracted. Let $\mathbf{i}$ be the mean, $\mathbf{i}$ ... and $\mathbf{i}$ be the neighborhood of a point $i$ in the image to be extracted. The domain size is consistent. It can be adjusted according to actual needs. There are many other methods for low-frequency extraction, such as Total Variation Image Denoising (TV), Local Edge Preserving Filter (LEP), Weighted Least Squares Filter (WLS), Bilateral Filter, etc. The choice can be made based on the actual use case and the processing performance of the cloud device.

[0048] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating a sub-process of step S12 in an embodiment of the image information extraction method of this application. For example... Figure 3 As shown, in some embodiments, step S12 above may include the following steps:

[0049] Step S121: Divide the image to be extracted into several first image blocks, and divide the target image into several second image blocks.

[0050] In this embodiment, each first image block and each second image block correspond one-to-one, and each first image block and its corresponding second image block form an image group. For example, the image to be extracted and the target image are divided in the same way, resulting in the same number of first image blocks and second image blocks, so that each first image block corresponds to one second image block, and different first image blocks correspond to different second image blocks. In some embodiments, the size and number of first and second image blocks in each image group are the same. Furthermore, the image block division method of the target image is the same as that of the image to be extracted; this embodiment uses the image block division of the image to be extracted as an example. For a better understanding of the image block division method, please refer to... Figure 4 , Figure 4 This is a schematic diagram illustrating the image block division result of an embodiment of the image information extraction method of this application. For example... Figure 4 As shown, the size of the image patch can be 16x16, 32x32, 64x64, or 16x32, 16x64, etc. In some application scenarios, the minimum length of the image patch is generally 16, and the maximum length is 64. If the image is not large enough to be divided into the last column and the last row, edge padding can be performed. The edge padding method for the image to be extracted and the target image are the same. The edge padding method can be to fill with preset pixel values ​​or to fill with dilation or other methods. The edge padding method can be any of the current edge padding methods, and no specific limitation is made here. Figure 4 In this process, the image to be extracted is divided into 20 blocks. These first image blocks may or may not overlap; this embodiment assumes no overlap between the first image blocks. In some embodiments, the first image blocks and the second image blocks do not contain the same pixel regions.

[0051] Step S122: For each image group, determine the mapping parameters corresponding to the image group based on the information correlation between the first image block and the second image block within the image group.

[0052] The set of mapping parameters corresponding to each image group is the mapping parameter between the image to be extracted and the target image.

[0053] In this embodiment, it is assumed that the information in the second image block of the target image is correlated with the information in the first image block of the image to be extracted, and a correlation energy function can be constructed based on this correlation. The correlation energy function is shown in formula (4).

[0054]

[0055] Where (i,j) are the image patch coordinates, id is the coordinate of a pixel within the image patch, and size is the number of pixels contained in the image patch. For example, for a 16x16 patch, size is 256. D(ClowUI) i,j ) is the target image CLowUI i,j The variance of the image patch (i,j). UI i,j,jd It is the pixel value of the coordinate point id in the first image patch, CLowUI i,j,id This is the pixel value of coordinate point id in the first image patch. θ and β are external configuration parameters that adjust the mapping intensity; the larger the value, the smoother the mapping. These external configuration parameters can be determined according to requirements. λ is introduced to prevent D(ClowUI) from being applied. i,j The case where 0 occurs represents a very small number, such as 0.01. The mapping parameters include parameters a, b, and c.

[0056] In some embodiments, the mapping parameters are matrices. These matrices may include multiple elements, for example, the elements being the parameters a, b, and c described above. Based on this, step S122 may include the following steps:

[0057] First, a first matrix, a second matrix, and a third matrix are determined for the image group. The first matrix is ​​obtained from the pixel values ​​of each first pixel in the first image block of the image group. The second matrix is ​​obtained from the statistical pixel values ​​of each second pixel in the second image block of the image group. The third matrix is ​​obtained from the pixel values ​​of each second pixel. Then, based on the first, second, and third matrices, the mapping parameters corresponding to the image group are determined.

[0058] Specifically, the first matrix is ​​obtained from the pixel values ​​of each first pixel in the first image block of the image group. Specifically, at least some elements of the first matrix may be related to the pixel values ​​of each first pixel. For example, at least some elements are the pixel values ​​of some first pixels.

[0059] The second matrix is ​​obtained from the pixel statistics of each second pixel in the second image block of the image group. Specifically, it can be obtained by statistically analyzing the pixel values ​​of each second pixel, and then determining at least some elements of the second matrix based on these pixel statistics. The pixel statistics can be any statistical value such as mean, variance, mode, median, etc.

[0060] The third matrix is ​​derived from the pixel values ​​of each second pixel. Specifically, at least some elements of the third matrix may be related to the pixel values ​​of each second pixel. For example, at least some elements are the pixel values ​​of some second pixels.

[0061] In some embodiments, the first matrix includes several rows, and different elements in each row correspond to different pixel data of a first pixel. Each pixel data element is the square of the pixel value, the pixel value itself, and a first preset value. The number of pixel data elements is related to the number of elements in the mapping parameters. Optionally, the number of rows in the first matrix is ​​related to the number of first pixels. The first preset value can be 1.

[0062] The second matrix is ​​a diagonal matrix. The number of rows and columns of the second matrix is ​​related to the number of elements contained in the mapping parameters. For example, if the mapping parameters are a matrix containing three elements a, b, and c, then the number of rows and columns of the second matrix is ​​greater than or equal to 3. This embodiment takes the example where the number of rows and columns of the second matrix is ​​the same as the number of elements contained in the mapping parameters. The elements in the first row and first column of the second matrix are obtained by the square root of the quotient of the second preset value and the pixel statistics value. The elements in the second row and second column of the second matrix are obtained by the square root of the quotient of the third preset value and the pixel statistics value. The elements in the N rows and N columns of the second matrix are the fourth preset value, where N is greater than 2. The fourth preset value can be 0.

[0063] The third matrix consists of several rows, each containing one element. Each element in the row corresponds to the pixel value of a second pixel.

[0064] In some application scenarios, the correlation energy function above can be matrixed and expressed as formula (5):

[0065]

[0066] Here, A is a 3x1 vector containing the parameters a, b, and c to be determined; A can be considered a matrix of mapping parameters. The third matrix M is a size×1 vector, with each row containing a CLowUI. i,j Elements. The first matrix H is a size × 3 matrix, and the second matrix D is a diagonal matrix.

[0067] Wherein, the matrix of mapping parameters

[0068] First matrix

[0069] Second matrix

[0070] Third matrix

[0071] The second preset value is θ, and the third preset value is β.

[0072] In some embodiments, the method for determining the mapping parameters corresponding to the image group based on the first matrix, the second matrix, and the third matrix can be as follows: First, obtain the inverse matrix of the sum of the first product and the second product. The first product is the product of the transpose of the first matrix and the first matrix, and the second product is the product of the transpose of the second matrix and the second matrix. Then, the fourth product obtained by multiplying the inverse matrix and the third product is used as the mapping parameter corresponding to the image group, where the third product is the product of the transpose of the first matrix and the third matrix.

[0073] Specifically, by differentiating A in formula (5), we obtain formula (6):

[0074]

[0075] The superscript T indicates transpose. The parameters a, b, and c for each image group can be determined using formula (6).

[0076] In some application scenarios, for a terminal device downloading single-channel 8-bit data, if the data volume required to transmit the target image is H×W×8 bits, the mapping parameters are represented using 32-bit floating-point, and the target image is divided into 32×32 image blocks, then the data volume required to transmit the mapping parameters is: It is clear that the difference in data volume between the two is significant, which greatly reduces the data transmission resource overhead between cloud devices and terminal devices.

[0077] In some applications, a downsampled image is transmitted to a cloud device, where it is upsampled. Low-frequency data is extracted from the upsampled image, and mapping parameters between the upsampled image and the low-frequency image are obtained using the principle of correlation between local image information. These mapping parameters are then transmitted back to the terminal device. Finally, the terminal device uses the mapping parameters to obtain an approximate low-frequency image of the original image, thus reducing the amount of data transmission.

[0078] In this embodiment, the executing entity of the image information extraction method can be a first image information extraction device. For example, the image information extraction device can be a cloud device, a server, or other processing equipment. The terminal device can be a monitoring device in a security system, a network video recorder, user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the image information extraction method can be implemented by a processor calling computer-readable instructions stored in memory.

[0079] Please see Figure 5 , Figure 5 This is a schematic flowchart of another embodiment of the image information extraction method of this application. Figure 5 As shown, the image information extraction method provided in this embodiment may include the following steps:

[0080] Step S21: The terminal device sends the image to be extracted to the cloud device.

[0081] The execution subject in this embodiment is a terminal device.

[0082] Step S22: Receive the mapping parameters sent by the cloud device.

[0083] Mapping parameters represent the mapping relationship between the target image and the image to be extracted. The amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image. The target image is obtained by the cloud device from the image to be extracted, and the target image contains target information. The method by which the cloud device obtains the mapping parameters can be referred to the above embodiment, and will not be repeated here.

[0084] Step S23: Process the image to be extracted using the mapping parameters to obtain the target image.

[0085] The mapping parameter can represent the mapping relationship between the image to be extracted and the target image, so the target image can be determined based on this mapping parameter and the image to be extracted.

[0086] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device. The cloud device then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, enabling it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device, allowing for the use of more powerful algorithms to extract information from the image, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to transmit the mapping parameter is less than the amount of data required to transmit the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0087] In some embodiments, the following steps may also be performed before performing step S21 described above:

[0088] The image to be extracted is downsampled to obtain a downsampled image. Then, the downsampled image is sent to a cloud device. The downsampled image is used by the cloud device for upsampling to obtain the image to be extracted. The downsampling ratio can be a preset ratio. There are many downsampling methods; please refer to the above embodiments for details, which will not be elaborated here.

[0089] For the terminal device, if the image to be extracted is uploaded directly, uploading 8-bit data from a single channel, the total required data volume is H×W×8 bits. If downsampling is performed 32 times in both the length and width directions, the data volume required to transmit the downsampled image is... It is clear that the difference in data volume between the two is significant, which greatly reduces the data transmission resource overhead between cloud devices and terminal devices.

[0090] In some embodiments, the mapping parameters include mapping parameters corresponding to several image groups. Each image group includes a first image block and a second image block. The first image block is obtained by dividing the image to be extracted, and the second image block is obtained by dividing the target image. The specific method of dividing the image blocks can be found in the above embodiments, and will not be repeated here.

[0091] Step S23 above may include the following steps: upsampling the mapping parameters to obtain upsampled mapping parameters. The upsampled mapping parameters represent the mapping relationship between each pixel in the image to be extracted and each pixel in the target image. Then, the image to be extracted is processed using the upsampled mapping parameters to obtain the target image.

[0092] Specifically, the mapping parameters received by the terminal are the mapping parameters between image blocks. Each image block contains multiple pixels. By upsampling the mapping parameters, the mapping parameters between each pixel can be obtained. The upsampling method of the mapping coefficients is the same as the image upsampling method. The mapping parameters of each pixel after upsampling are ca, cb, and cc, respectively. The length and width of the mapping parameters are consistent with the size of the image to be extracted. Specifically, the formula for processing the image to be extracted I using the upsampled mapping parameters to obtain the target image LowI can be found in formula (7).

[0093] LowI = ca × I 2 +cb×I+cc formula (7);

[0094] Where LowI represents the target image, I represents the image to be extracted, and the mapping parameters include ca, cb, and cc. Of course, there may be some error in the mapping parameters during the upsampling process, leading to a slight difference between the target image determined at the terminal device and the target image obtained by the cloud device. However, this difference is small and can be ignored.

[0095] In this embodiment, the executing entity of the image information extraction method can be a second image information extraction device. For example, the image information extraction device can be a terminal device, a server, or other processing equipment. The terminal device can be a monitoring device in a security system, a network video recorder, user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the image information extraction method can be implemented by a processor calling computer-readable instructions stored in memory.

[0096] Please see Figure 6 , Figure 6 This diagram illustrates the interaction between a cloud device and a terminal device, representing another embodiment of the image information extraction method of this application. Figure 6 As shown, the image information extraction method provided in this embodiment may include the following steps:

[0097] Step S31: The terminal device sends the image to be extracted to the cloud device.

[0098] The image information processing method provided in this embodiment can be applied to image information extraction systems that include cloud devices and terminal devices.

[0099] Step S32: The cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing target information.

[0100] The specific method by which the cloud device extracts information from the image to be extracted sent by the terminal device to obtain the target image containing the target information can be referred to the relevant description in the above embodiment where the execution device is a cloud device, and will not be repeated here.

[0101] Step S33: The cloud device determines the mapping parameters based on the image information in the image to be extracted and the target image.

[0102] The mapping parameters represent the mapping relationship between the target image and the image to be extracted. The amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image. Furthermore, the specific method by which the cloud device determines the mapping parameters based on the image information in the target and the image to be extracted can be found in the relevant description in the embodiment where the execution device is a cloud device, and will not be repeated here.

[0103] Step S34: The cloud device sends the mapping parameters to the terminal device.

[0104] Step S35: The terminal device processes the image to be extracted using the mapping parameters to obtain the target image.

[0105] The terminal device processes the image to be extracted using mapping parameters, and the method for obtaining the target image can be referred to the relevant description in the above embodiment where the execution device is a terminal device, which will not be repeated here.

[0106] The specific implementation process of the image information extraction method in the information extraction system can be found in the embodiments where the execution device is a cloud device and the embodiments where the execution device is a terminal device, and will not be repeated here.

[0107] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device. The cloud device then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, allowing it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device, enabling the use of more powerful algorithms for information extraction, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0108] Please see Figure 7 , Figure 7This is a schematic diagram of an embodiment of the first image information extraction device of this application. The first image information extraction device 40 includes an information extraction module 41, a mapping parameter determination module 42, and a communication module 43. The information extraction module 41 is used by the cloud device to extract information from the image to be extracted sent by the terminal device to obtain a target image containing target information; the mapping parameter determination module 42 is used to determine mapping parameters based on the image information in the image to be extracted and the target image, the mapping parameters being used to represent the mapping relationship between the target image and the image to be extracted; the communication module 43 is used to send the mapping parameters to the terminal device, the terminal device being able to process the image to be extracted based on the mapping parameters to obtain the target image; wherein, the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0109] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device. The cloud device then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, allowing it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device, enabling the use of more powerful algorithms for information extraction, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0110] The functions of each module can be found in the embodiment of the image information extraction method, and will not be repeated here.

[0111] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the second image information extraction device of this application. The second image information extraction device 50 includes an information sending module 51, an information receiving module 52, and a processing module 53. The information sending module 51 is used by the terminal device to send the image to be extracted to the cloud device; the information receiving module 52 is used to receive mapping parameters sent by the cloud device, the mapping parameters being used to represent the mapping relationship between the target image and the image to be extracted, the target image being obtained by the cloud device from the image to be extracted and containing target information; the processing module 53 is used to process the image to be extracted using the mapping parameters to obtain the target image; wherein, the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image.

[0112] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device. The cloud device then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, allowing it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device, enabling the use of more powerful algorithms for information extraction, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0113] The functions of each module can be found in the embodiment of the image information extraction method, and will not be repeated here.

[0114] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the cloud device of this application. The cloud device 60 includes a first memory 61 and a first processor 62. The first processor 62 is used to execute program instructions stored in the first memory 61 to implement the steps in any of the above-described image information extraction method embodiments. In a specific implementation scenario, the cloud device 60 may include, but is not limited to, monitoring equipment, microcomputers, and servers. In addition, the cloud device 60 may also include mobile devices such as laptops and tablets, which are not limited here.

[0115] Specifically, the first processor 62 controls itself and the first memory 61 to implement the steps in any of the above-described image information extraction method embodiments. The first processor 62 can also be referred to as a CPU (Central Processing Unit). The first processor 62 may be an integrated circuit chip with signal processing capabilities. The first processor 62 can also be a general-purpose first processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose first processor can be a micro-first processor or any conventional first processor. Furthermore, the first processor 62 can be implemented using integrated circuit chips.

[0116] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device 60. The cloud device 60 then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, enabling it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device side, allowing for the use of more powerful algorithms to extract information from the image, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device 60 and the terminal device.

[0117] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of an embodiment of the terminal device of this application. The terminal device 70 includes a second memory 71 and a second processor 72. The second processor 72 is used to execute program instructions stored in the second memory 71 to implement the steps in any of the above-described image information extraction method embodiments. In a specific implementation scenario, the terminal device 70 may include, but is not limited to, monitoring equipment, microcomputers, and servers. In addition, the terminal device 70 may also include mobile devices such as laptops and tablets, which are not limited here.

[0118] Specifically, the second processor 72 controls itself and the second memory 71 to implement the steps in any of the above-described image information extraction method embodiments. The second processor 72 can also be referred to as a CPU (Central Processing Unit). The second processor 72 may be an integrated circuit chip with signal processing capabilities. The second processor 72 can also be a general-purpose second processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose second processor can be a micro second processor or any conventional second processor. Furthermore, the second processor 72 can be implemented using integrated circuit chips.

[0119] In the above scheme, after receiving the image to be extracted from the terminal device 70 and sending it to the cloud device, the cloud device extracts information from the image to obtain a target image containing the target information. Then, based on the image information between the image to be extracted and the target image, a mapping parameter between them is determined, and this mapping parameter is sent to the terminal device 70. Upon receiving the mapping parameter, the terminal device 70 can process the image to be extracted based on the mapping parameter to obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device 70 side, allowing the use of more powerful algorithms to extract information from the image to be extracted, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device and the terminal device 70.

[0120] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of an embodiment of the image information extraction system of this application. For example... Figure 11 As shown, the image information extraction system 1 includes a cloud device 60 and a terminal device 70 as described in the above embodiment. The terminal device 70 interacts with the cloud device 60 to implement any of the image information extraction methods described above.

[0121] In the above scheme, after receiving the image to be extracted from the terminal device 70 and sending it to the cloud device 60, the cloud device 60 extracts information from the image to obtain a target image containing the target information. Then, based on the image information between the image to be extracted and the target image, it determines the mapping parameters between them and sends these mapping parameters back to the terminal device 70. Upon receiving the mapping parameters, the terminal device 70 can process the image to be extracted based on these parameters to obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device 70, allowing for the use of more powerful algorithms to extract information from the image, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device 60 and the terminal device 70.

[0122] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor. The program instructions 81 are used to implement the steps in any of the above-described embodiments of the image information extraction method.

[0123] The above scheme involves receiving the image to be extracted from the terminal device and sending it to the cloud device. The cloud device then extracts information from the image to obtain a target image containing the target information. Based on the image information between the image to be extracted and the target image, a mapping parameter is determined between them. This mapping parameter is then sent to the terminal device, allowing it to process the image to be extracted and obtain the target image. This scheme eliminates the need to deploy the image extraction algorithm on the terminal device, enabling the use of more powerful algorithms for information extraction, resulting in a more accurate target image containing the target information. Furthermore, the amount of data required to send the mapping parameter to the terminal device is less than the amount of data required to send the target image, improving the transmission efficiency between the cloud device and the terminal device.

[0124] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0125] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for extracting image information, characterized in that, include: The cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing the target information; Based on the image information in the image to be extracted and the target image, a mapping parameter is determined, which is used to represent the mapping relationship between the target image and the image to be extracted; The mapping parameters are sent to a terminal device, which can process the image to be extracted based on the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image; The step of determining the mapping parameters based on the image information in the image to be extracted and the target image includes: dividing the image to be extracted into several first image blocks and dividing the target image into several second image blocks, with each first image block and each second image block corresponding one-to-one, and each first image block and its corresponding second image block forming an image group; for each image group, determining the mapping parameters corresponding to the image group based on the information correlation between the first image blocks and the second image blocks within the image group, and the set of mapping parameters corresponding to each image group is the mapping parameters between the image to be extracted and the target image.

2. The method according to claim 1, characterized in that, The mapping parameters are matrices. For each image group, the mapping parameters corresponding to the image group are determined based on the information correlation between the first image block and the second image block within the image group, including: A first matrix, a second matrix, and a third matrix are determined for the image group. The first matrix is ​​obtained from the pixel values ​​of each first pixel in the first image block of the image group. The second matrix is ​​obtained from the statistical pixel values ​​of each second pixel in the second image block of the image group. The third matrix is ​​obtained from the pixel values ​​of each second pixel. Based on the first matrix, the second matrix, and the third matrix, the mapping parameters corresponding to the image group are determined.

3. The method according to claim 2, characterized in that, The first matrix includes several rows, and different elements in each row correspond to different pixel data of a first pixel point. Each pixel data is the square of the pixel value, the pixel value, and a first preset value. The number of pixel data is related to the number of elements in the mapping parameters. The second matrix is ​​a diagonal matrix. The number of rows and columns of the second matrix is ​​related to the number of elements contained in the mapping parameters. The elements in the first row and first column of the second matrix are obtained by the square root of the quotient of the second preset value and the pixel statistics value. The elements in the second row and second column of the second matrix are obtained by the square root of the quotient of the third preset value and the pixel statistics value. The elements in the N rows and N columns of the second matrix are the fourth preset value, where N is greater than 2. The third matrix comprises several rows, each row containing one element, and each element in the row corresponds to the pixel value of a second pixel.

4. The method according to claim 2 or 3, characterized in that, The step of determining the mapping parameters corresponding to the image group based on the first matrix, the second matrix, and the third matrix includes: Obtain the inverse matrix of the sum of the first product and the second product, where the first product is the product of the transpose of the first matrix and the first matrix, and the second product is the product of the transpose of the second matrix and the second matrix; The fourth product, obtained by multiplying the inverse matrix by the third product, is used as the mapping parameter corresponding to the image group. The third product is the product of the transpose of the first matrix and the third matrix.

5. The method according to any one of claims 1-3, characterized in that, In each of the image groups, the first image block and the second image block have the same size and number, and / or, the first image blocks do not contain the same pixel regions, and the second image blocks do not contain the same pixel regions.

6. The method according to claim 1, characterized in that, Before the cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing target information, the method further includes: Receive a downsampled image sent by the terminal device, wherein the downsampled image is obtained by downsampling the image to be extracted by the terminal device; The downsampled image is upsampled to obtain the image to be extracted.

7. The method according to claim 1, characterized in that, The cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing target information, including: Using the image to be extracted as a guide image, guided filtering is performed on the image to be extracted to obtain an image containing low-frequency information, and the image containing low-frequency information is the target image.

8. A method for extracting image information, characterized in that, The terminal device sends the image to be extracted to the cloud device; The system receives mapping parameters sent by a cloud device. These mapping parameters represent the mapping relationship between a target image and the image to be extracted. The target image is obtained by the cloud device through information extraction from the image to be extracted, and the target image contains target information. The image to be extracted is processed using the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image; The mapping parameters are obtained as follows: the cloud device divides the image to be extracted into several first image blocks and the target image into several second image blocks, with each first image block and each second image block corresponding one-to-one, and each first image block and its corresponding second image block forming an image group; for each image group, based on the information correlation between the first image blocks and the second image blocks within the image group, the mapping parameters corresponding to the image group are determined, and the set of mapping parameters corresponding to each image group is the mapping parameters between the image to be extracted and the target image.

9. The method according to claim 8, characterized in that, The terminal device sends the image to be extracted to the cloud device, including: The image to be extracted is downsampled to obtain a downsampled image; The downsampled image is sent to a cloud device, wherein the downsampled image is provided to the cloud device for upsampling to obtain the image to be extracted.

10. The method according to claim 8, characterized in that, The mapping parameters include mapping parameters corresponding to several image groups. The step of processing the image to be extracted using the mapping parameters to obtain the target image includes: The mapping parameters are upsampled to obtain upsampled mapping parameters, which represent the mapping relationship between each pixel in the image to be extracted and each pixel in the target image. The image to be extracted is processed using the upsampled mapping parameters to obtain the target image.

11. A method for extracting image information, characterized in that, include: The terminal device sends the image to be extracted to the cloud device; The cloud device extracts information from the image to be extracted sent by the terminal device to obtain a target image containing target information; The cloud device determines mapping parameters based on image information in the image to be extracted and the target image. The mapping parameters are used to represent the mapping relationship between the target image and the image to be extracted. The cloud device sends the mapping parameters to the terminal device; The terminal device processes the image to be extracted using the mapping parameters to obtain the target image; wherein the amount of data required to send the mapping parameters to the terminal device is less than the amount of data required to send the target image. The cloud device determines mapping parameters based on image information in the image to be extracted and the target image, including: dividing the image to be extracted into several first image blocks and dividing the target image into several second image blocks, with each first image block and each second image block corresponding one-to-one, and each first image block and its corresponding second image block forming an image group; for each image group, determining the mapping parameters corresponding to the image group based on the information correlation between the first image blocks and the second image blocks within the image group, and the set of mapping parameters corresponding to each image group is the mapping parameters between the image to be extracted and the target image.

12. A cloud device, characterized in that, It includes a first memory and a first processor, the first processor being configured to execute program instructions stored in the first memory to implement the method according to any one of claims 1 to 7.

13. A terminal device, characterized in that, The method includes a second memory and a second processor, the second processor being configured to execute program instructions stored in the second memory to implement the method of any one of claims 8 to 10.

14. A system for extracting image information, characterized in that, The system includes the cloud device as described in claim 12 and the terminal device as described in claim 13.

15. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1 to 11.

Images