Image matching method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image matching method and device, electronic equipment and a storage medium. The method comprises the following steps: intercepting a to-be-matched image block from a to-be-processed image by using a sliding window; matching the feature value of the to-be-matched image block with a plurality of pre-stored feature values; if a target pre-stored feature value matched with the feature value of the to-be-matched image block exists, determining a target image block from the to-be-processed image according to the position of a pre-stored image block corresponding to the target pre-stored feature value in a target historical image, an integrogram of the target historical image and the position of the to-be-matched image block in the to-be-processed image, the target image block is a maximum matching image area in the to-be-processed image and the target historical image, and the target image block comprises the to-be-matched image block. Through the method, the maximum matching image area in the to-be-processed image and the target historical image can be quickly and accurately found, so that the image data transmission quantity of the to-be-processed image is effectively reduced.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and more particularly, to an image matching method, apparatus, electronic device, and storage medium. Background Art

[0002] The Virtual Desktop Infrastructure (VDI) is the mainstream architecture and deployment method in current desktop cloud solutions. The idea of VDI is to save the desktop in the data center through virtualization technology, and end users can access the desktop through the virtual desktop control protocol to obtain a complete computer usage experience. In the VDI architecture, the virtual desktop image content has a large amount of data and requires a very high bandwidth.

[0003] In the related art, the method of transmitting virtual desktop images is a fixed block transmission method. The idea of the fixed block transmission method is to divide each virtual desktop image into blocks of a fixed size, and then query whether the hash value of the block has been stored in the cache. If the hash value of the block already exists in the cache, only the hash value and location of the block are sent to the client, and the client copies the block data from the cache to restore the block. If the block does not exist in the cache, the block is sent to the client, and the client decodes to obtain the block.

[0004] Obviously, the method in the related art can reduce the server transmission volume to a certain extent and reduce the bandwidth requirement. However, since the size of the image block is usually set based on the experience of R & D personnel, when the image block is small, it usually leads to a high search complexity. When the image block is large, the probability of finding the hash value of the block in the cache is low. That is to say, the number of blocks directly sent from the server to the client will be relatively large, and a large bandwidth is also required to meet the need of transmitting virtual desktop images. Therefore, it is an urgent technical problem to provide a method that can quickly find the largest matching image block. Summary of the Invention

[0005] In view of this, the embodiments of this application propose an image matching method, apparatus, electronic device, and storage medium, which can quickly and accurately find the largest matching image area between the image to be processed and the target historical image.

[0006] In a first aspect, an embodiment of the present application provides an image matching method, the method comprising: obtaining an image to be processed; using a sliding window to intercept a to-be-matched image block from the image to be processed; matching the eigenvalue of the to-be-matched image block with a plurality of pre-stored eigenvalues, each of the pre-stored eigenvalues corresponding to a pre-stored image block, the pre-stored image block being obtained by dividing a historical image, and the shape and size of the pre-stored image block being the same as those of the sliding window; if there is a target pre-stored eigenvalue that matches the eigenvalue of the to-be-matched image block, determining a target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored eigenvalue in the target historical image, the integral image of the target historical image, and the position of the to-be-matched image block in the image to be processed, the target image block being the largest matching image region between the image to be processed and the target historical image, and the target image block including the to-be-matched image block.

[0007] In a second aspect, an embodiment of the present application provides an image matching device, comprising: a data acquisition module, configured to obtain an image to be processed; an image block interception module, configured to use a sliding window to intercept a to-be-matched image block from the image to be processed; an eigenvalue matching module, configured to match the eigenvalue of the to-be-matched image block with a plurality of pre-stored eigenvalues, each of the pre-stored eigenvalues corresponding to a pre-stored image block, the pre-stored image block being obtained by dividing a historical image, and the shape and size of the pre-stored image block being the same as those of the sliding window; a region search module, configured to, when there is a target pre-stored eigenvalue that matches the eigenvalue of the to-be-matched image block, determine a target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored eigenvalue in the target historical image, the integral image of the target historical image, and the position of the to-be-matched image block in the image to be processed, the target image block being the largest matching image region between the image to be processed and the target historical image, and the target image block including the to-be-matched image block.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, and wherein the above method is executed when the program code is run by a processor.

[0010] Fifth aspect, an embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device obtains the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0011] An image matching method, device, electronic device and storage medium provided by an embodiment of the present application. By obtaining an image to be processed; using a sliding window to intercept an image block to be matched from the image to be processed; matching the feature value of the image block to be matched with a plurality of pre-stored feature values, each of the pre-stored feature values corresponds to a pre-stored image block respectively, and the pre-stored image block is obtained by dividing a historical image, and the shape and size of the pre-stored image block are the same as those of the sliding window; if there is a target pre-stored feature value that matches the feature value of the image block to be matched, determining a target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored feature value in the target historical image, the integral image of the target historical image, and the position of the image block to be matched in the image to be processed, the target image block is the largest matching image area between the image to be processed and the target historical image, and the target image block includes the image block to be matched. By using a sliding window to intercept the image block to be matched instead of a fixed block division method to obtain the image block to be matched, when using the feature value of the image block to be matched to match with the pre-stored feature value, the probability of matching the pre-stored feature value can be improved. When a matching target pre-stored feature value is found, further determine the target image block from the image to be processed, so as to identify the largest identical area from the image to be processed and the target historical image, thereby maximizing the use of cached data and reducing the amount of image data transmission of the image to be processed, so as to reduce the data transmission bandwidth. Description of the Drawings

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0013] Figure 1 Shows a timing process schematic diagram of an image matching method provided by an embodiment of the present application;

[0014] Figure 2 Shows a schematic diagram of a sliding window intercepting an image block to be matched in an image to be processed;

[0015] Figure 3 Shows Figure 1A schematic flowchart of step S140 in

[0016] Figure 4 A schematic diagram showing the maximum boundaries of a to-be-matched image block provided by an embodiment of the present application in each direction;

[0017] Figure 5 A schematic diagram showing a to-be-processed image and a target historical image provided by an embodiment of the present application;

[0018] Figure 6 Shows Figure 1 Another schematic flowchart of step S140 in

[0019] Figure 7 Another schematic flowchart of an image matching method provided by an embodiment of the present application;

[0020] Figure 8 Another schematic flowchart of an image matching method provided by an embodiment of the present application;

[0021] Figure 9 A connection block diagram of an image matching device provided by an embodiment of the present application;

[0022] Figure 10 Shows a structural block diagram of an electronic device for executing the method of the embodiment of the present application. Detailed implementation manners

[0023] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0024] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0025] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0027] It should be noted that: "a plurality of" as mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0028] In addition, it should be noted that in the embodiments of the present application, the collection, use, processing, and storage of application information are all subject to the user's permission and need to comply with the regulations of the region where it is located.

[0029] An image matching method provided by the present application can be applied to an electronic device, which can be a server, a terminal device, or a combination of one or more of the above.

[0030] In some embodiments, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0031] Figure 1 Specifically, the image matching method of the present application is shown. This method can be applied to an electronic device. Taking the electronic device as a server specifically, the method includes:

[0032] Step S110: Obtain the image to be processed.

[0033] Among them, the image to be processed can be any image content that needs to be transmitted from the server side to the client through VDI (Virtual Desktop Infrastructure). It can include but is not limited to: desktop background (i.e., the desktop wallpaper or background image set by the user), application interface (i.e., the screenshot of the running application window, such as a document editor, browser, multimedia player, etc.), graphics and icons (i.e., various shortcuts on the desktop, folder icons, and icons in the system tray), video frames (if the user is watching a video, then each frame of the video image can also be the image to be processed), and dynamically updated content (such as the new screen generated when the user scrolls the web page or interacts in the application), etc.

[0034] The way to obtain the image to be processed can be that the server can capture the currently displayed image content through a screenshot operation (i.e., in a VDI environment, the server usually has a virtual display device responsible for rendering the user's desktop environment and all graphic outputs, and the server can capture the image content currently displayed on this virtual display device through a screenshot operation). It can also be that the server calls an application programming interface to obtain the image data to be processed. It can also be to obtain the pre-stored image data to be processed in response to an image acquisition instruction sent by the terminal device.

[0035] The above ways of obtaining the image to be processed are only illustrative, and there can be more ways of obtaining, which are not specifically limited here.

[0036] Step S120: Intercept a to-be-matched image block from the image to be processed by using a sliding window.

[0037] Among them, the shape of the sliding window can be a regular shape such as a circle, rectangle, etc., or any irregular shape. In the application embodiment, the shape of the sliding window is a rectangle.

[0038] Before intercepting the to-be-matched image block from the image to be processed by using the sliding window for matching, usually a specific position of the image to be processed (such as the upper left corner, upper right corner, lower left corner, lower right corner of the image, or any position in the image) is determined as the starting sliding position of the sliding window.

[0039] It is worth mentioning that if the upper left corner of the image to be processed is the starting sliding position of the sliding window, at this time, the upper left corner of the sliding window coincides with the upper left corner of the image to be processed; at this time, the upper edge pixel points of the to-be-matched image block intercepted by the sliding window coincide with the upper edge pixel points of the image to be processed, and the left edge pixel points of the to-be-matched image block coincide with the left edge pixel points of the image to be processed. Similarly, if the upper right corner of the image to be processed is the starting sliding position of the sliding window, at this time, the upper right corner of the sliding window coincides with the upper right corner of the image to be processed.

[0040] During the sliding process of the sliding window, its shape and size do not change, and the step size of the sliding window is used to define the distance that the window moves in the horizontal and vertical directions each time. Among them, the step size can be equal to the window size (no overlap) or less than the window size (with overlap) to ensure a finer match. Before moving the sliding window each time, it is necessary to check whether the new window position exceeds the boundary of the image to be processed. If it exceeds the boundary, the position of the window needs to be adjusted or the sliding needs to be stopped.

[0041] Exemplarily, as Figure 2 shown, it shows that the image to be processed has four directions: up, down, left, and right. Starting from the upper left corner (coordinates (0, 0)) of the image to be processed, a sliding window of a predefined size (for example, a×b pixels) is used to slide and intercept the image block to be matched. It can be seen that before and after the sliding window slides, its size does not change, and the size of each intercepted image block to be processed is the same. Among them, a and b are positive integers respectively, representing the number of pixel points.

[0042] Step S130: Match the feature values of the image block to be matched with multiple pre-stored feature values.

[0043] Among them, each of the pre-stored feature values corresponds to a pre-stored image block, and the pre-stored image block is obtained by dividing a historical image. The shape and size of the pre-stored image block are the same as those of the sliding window.

[0044] The pre-stored feature value is obtained by performing feature extraction on the corresponding pre-stored image block.

[0045] The multiple pre-stored image blocks can be obtained by dividing multiple historical images into rectangular blocks of a fixed size, or by moving and dividing multiple historical images using the aforementioned sliding window according to a specified step size. No specific limitation is made here.

[0046] Among them, the feature value of the image block to be matched can be any one of the color histogram, texture feature, hash value, etc. of the image block to be processed, which can distinguish different image contents and at the same time describe the characteristics of the image block to be processed, a numerical value or vector.

[0047] It is worth mentioning that the type of the feature value of the image block to be matched is the same as the type of the pre-stored feature value, that is, when the feature value of the image block to be matched is a hash value, the pre-stored feature value is specifically a pre-stored hash value; when the feature value of the image block to be matched is a texture feature, the pre-stored feature value is specifically a pre-stored texture feature.

[0048] When the eigenvalue is a hash value, the method of matching the eigenvalue of the image block to be matched with multiple pre-stored eigenvalues can be as follows: calculate the Hamming distance between the hash value of the image block to be matched and each pre-stored hash value respectively to obtain the distance value corresponding to each pre-stored hash value. If there is a distance value corresponding to a pre-stored hash value that is less than the preset Hamming distance threshold, it is determined that the pre-stored hash value matches the hash value of the image block to be matched, that is, the pre-stored hash value is the target pre-stored eigenvalue.

[0049] When the eigenvalue is a color histogram, the method of matching the eigenvalue of the image block to be matched with multiple pre-stored eigenvalues can be as follows: calculate the similarity (such as chi-square distance or cross-entropy, etc.) between the color histogram of the image block to be matched and each pre-stored color histogram to obtain the similarity corresponding to each pre-stored color histogram; if there is a similarity corresponding to a pre-stored color histogram that is lower than the preset similarity threshold, it is determined that the pre-stored image block corresponding to the pre-stored color histogram matches the image block to be matched.

[0050] When the eigenvalue is a texture feature, the method of matching the eigenvalue of the image block to be matched with multiple pre-stored eigenvalues can be as follows: calculate the difference degree (such as Euclidean distance, Manhattan distance, cosine similarity) between the texture feature of the image block to be matched and each pre-stored texture feature to obtain the difference degree corresponding to each pre-stored texture feature. If there is a difference degree corresponding to a pre-stored texture feature that is lower than the preset difference degree threshold, it is determined that the pre-stored image block corresponding to the pre-stored texture feature matches the image block to be matched.

[0051] It should be understood that the above matching methods are only illustrative, and there can be more matching methods, which will not be elaborated here one by one.

[0052] If there is a target pre-stored eigenvalue that matches the eigenvalue of the image block to be matched, execute step S140: determine the target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored eigenvalue in the target historical image, the integral image of the target historical image, and the position of the image block to be matched in the image to be processed.

[0053] Among them, the target image block is the largest matching image area between the image to be processed and the target historical image, and the target image block includes the image block to be matched.

[0054] The integral image of a historical image, also known as the Summed Area Table, is a data structure used to accelerate image processing tasks. It is mainly used to quickly calculate the sum of pixel values within an arbitrary rectangular region. For a given two-dimensional grayscale image I(x, y), its corresponding integral image S(x, y) is defined as the cumulative sum of all pixel values from the top-left corner (0, 0) of the image to the point (x, y). This means that each element in the integral image represents the sum of all pixel values within the rectangular region from the top-left corner to the current position in the original image.

[0055] In one implementable manner, the above step S140 may be to extract the first integral value of the first adjacent image block of the pre-stored image block according to the position of the pre-stored image block corresponding to the target pre-stored eigenvalue in the target historical image and the integral image of the target historical image. The first adjacent image block is adjacent to the pre-stored image block, and the number of pixels in the first adjacent image block is less than the number of pixels in the pre-stored image block; obtain the second integral value of the second adjacent image block corresponding to the first adjacent image block in the image to be processed; if there is a match between the first integral value of the first adjacent image block and the second integral value of the corresponding second adjacent image block, add the second adjacent image block to the image blocks to be matched to obtain a new image block to be matched, and add the first vector image block to the target historical image block to obtain a new target historical image block, and return to execute the step of extracting the first integral value of the first adjacent image block of the pre-stored image block according to the position of the pre-stored image block corresponding to the target pre-stored eigenvalue in the target historical image and the integral image of the target historical image, until there is no match between the first integral value of the first adjacent image block and the second integral value of the corresponding second adjacent image block, and determine the last obtained new image block to be matched as the target image block.

[0056] By using the integral image of the historical image, the integral value of any image block in the historical image can be quickly obtained.

[0057] By adopting the image matching method of the present application, when obtaining the image blocks to be matched by using a sliding window to intercept the image blocks to be matched instead of a fixed partitioning method, and using the eigenvalue of the image block to be matched to match with the pre-stored eigenvalue, the probability of matching the pre-stored eigenvalue can be improved. When a matching target pre-stored eigenvalue is found, further determine the target image block from the image to be processed, so as to identify the largest identical region from the image to be processed and the target historical image, thereby maximizing the use of cached data, reducing the amount of image data transmission of the image to be processed, and reducing the data transmission bandwidth.

[0058] In one implementable manner, please refer to Figure 3, the above step S140 may be: using the integral image of the target historical image and the position of the image block to be matched corresponding to the target historical image, expanding the image block to be matched in at least one of the four directions of up, down, left, and right of the image to be processed to obtain a target image block.

[0059] In this embodiment, the above step S140 may specifically include the following steps:

[0060] Step S1401: Select one direction from the initial direction group as the first direction, where the initial direction group includes the four directions of up, down, left, and right.

[0061] Step S1402: Move the first side of the image block to be matched to the maximum boundary in the first direction to obtain a first candidate image block.

[0062] Wherein, the first side is the side parallel to the maximum boundary in the first direction and with the shortest distance.

[0063] The maximum boundary in the first direction may be the boundary of the image to be processed in the first direction, or may be determined according to the preset movement rule of the sliding window and the size of the sliding window.

[0064] Hereinafter, taking the maximum boundary in the first direction being determined according to the preset movement rule of the moving window and the size of the sliding window as an example for illustration:

[0065] In an implementable manner, when the preset movement rule indicates that the sliding window jumps and moves at preset pixels from left to right along the image to be processed and when the right side edge pixel of the image block to be matched intercepted by the sliding window is the right side edge pixel of the image to be processed, move the sliding window down one pixel along the image to be processed, and control the left side edge of the sliding window to move to the left side edge pixel of the image to be processed, so as to perform jump movement at preset pixels from left to right along the image to be processed subsequently.

[0066] In this case, when the first direction is upward, the distance between the maximum boundary in the first direction and the upper edge pixel of the image block to be matched is the height of the sliding window; when the first direction is downward, the maximum boundary in the first direction is the lower edge of the image to be processed; when the first direction is leftward, the distance between the maximum boundary in the first direction and the left edge pixel of the image block to be matched is the width of the sliding window; when the first direction is rightward, the maximum boundary in the first direction is the right edge of the image to be processed.

[0067] Exemplarily, such as Figure 4As shown, a sliding window and the maximum boundaries of the sliding window in the four directions of up, down, left, and right are shown. Among them, when the size of the sliding window is a×b and the above-mentioned preset sliding rule is used for sliding, the maximum boundary in the up direction is parallel to the upper edge of the image block to be matched and is b pixels away from the upper side edge; the maximum boundary in the down direction is the lower edge of the image to be processed; the maximum boundary in the left direction is parallel to the left edge of the image block to be matched and is a pixel points away from the left edge, and the maximum boundary in the right direction is the right edge of the image to be processed.

[0068] In another feasible implementation, when the preset movement rule indicates that the sliding window makes a jump movement at a preset pixel interval from right to left along the image to be processed and when the left side edge pixel point of the image block to be matched intercepted by the sliding window is the left edge pixel point of the image to be processed, move the sliding window one pixel point from top to bottom along the image to be processed, and control the right edge of the sliding window to move to the left edge pixel point of the image to be processed, so as to make a jump movement at a preset pixel interval from right to left along the image to be processed subsequently.

[0069] In this case, when the first direction is up, the distance between the maximum boundary in the first direction and the upper edge pixel point of the image block to be matched is the height of the sliding window; when the first direction is down, the maximum boundary in the first direction is the lower edge of the image to be processed; when the first direction is left, the maximum boundary in the first direction is the left edge of the image to be processed; when the first direction is right, the distance between the maximum boundary in the first direction and the right edge pixel point of the image block to be matched is the width of the sliding window.

[0070] It should be understood that the above method for determining the maximum boundary in the first direction according to the preset movement rule of the moving window and the size of the sliding window is only illustrative, and there can be more determination methods.

[0071] By adopting the above setting method, it is possible to avoid re-matching the already matched area, thereby effectively saving computing resources.

[0072] Step S1403: Determine whether the dichotomy stop condition is satisfied.

[0073] Among them, the stop condition of the dichotomy can be when a preset number of iterations is reached, or when the search range is reduced to a preset accuracy (for example, less than a preset pixel), stop the dichotomous search.

[0074] In the embodiment, the dichotomy stop condition can be that the search range is reduced to one pixel.

[0075] If the bisection stopping condition is satisfied, step S1404 is executed: determining a first reference feature value according to the integral image of the target historical image and the position of the first candidate image block in the target historical image.

[0076] Regarding the process of determining the reference feature value according to the integral image of the target historical image and the position of the candidate image block in the target historical image, reference can be made to the foregoing description of calculating the integral value of the image block in step S140, which will not be elaborated here one by one.

[0077] Step S1405: obtaining a first actual feature value of the first candidate image block according to the pixel parameters of each pixel in the first candidate image block.

[0078] Wherein, the pixel parameter of the pixel can specifically be the pixel value. Regarding obtaining the first actual feature value of the first candidate image block according to the pixel parameters of each pixel in the first candidate image block, reference can also be made to the foregoing specific description of calculating the integral value of the image block in step S140, which will not be elaborated here one by one.

[0079] Step S1406: according to the matching result between the first actual feature value and the first reference feature value, using the bisection method to determine a reference boundary based on the maximum boundary of the image block to be matched on the first side and the first direction of the image to be processed.

[0080] Wherein, the non - matching between the first actual feature value and the first reference feature value may mean that the difference between the first actual feature value and the second reference feature value is greater than the first preset difference threshold. Correspondingly, when the difference between the first actual feature value and the first reference feature value is not greater than the preset difference threshold, the first actual feature value and the first reference feature value match. The specific value of the first preset difference threshold can be set according to actual needs, and is not specifically limited in this embodiment.

[0081] Step S1407: taking the reference boundary as the new maximum boundary, and returning to execute step 1402. Until the bisection stopping condition is reached, step S1408 is executed: taking the first candidate image block as the new image block to be matched, deleting the first direction from the initial direction group; and executing step S1409: determining whether there is a direction in the initial direction group. If there is, return to step S1401. Until the first actual feature value matches the first reference feature value and there is no direction in the initial direction group, step S1410 is executed: taking the first candidate image block corresponding to the first actual feature value as the target image block.

[0082] Through the above steps S1401 - S1410, when the target image is found, by performing expansion attempts in the four directions of up, down, left, and right respectively, it is ensured that the largest matching region that best meets the requirements is found, rather than being limited to a specific direction. Once all possible directions have been tried and no new matches appear, the algorithm automatically stops, avoiding the problem of incorrect matching caused by over-expansion. In addition, during the matching process, by adopting the binary matching method, the computational amount can be effectively reduced, and the target image block can be quickly found.

[0083] As Figure 5 shown, a schematic diagram of the image to be processed (a) and the target historical image (b) is shown. By adopting the foregoing method steps, after the image block to be matched is first obtained using the sliding window, the target pre-stored image block that matches the image block to be matched can be found from multiple pre-stored image blocks corresponding in the target historical image. Thereafter, the target image block can be obtained by adopting the foregoing steps S1401 - S1410.

[0084] Among them, the target image block is the largest matching image region between the image to be processed and the target historical image. Correspondingly, there is an image block in the target historical image that matches the target image block.

[0085] In another implementable manner, please refer to Figure 6 shown, the above step S140 further includes:

[0086] Step S1411: Select one direction from the initial direction group as the second direction.

[0087] Among them, the initial direction group includes the four directions of up, down, left, and right.

[0088] Step S1412: Determine whether the second side of the image block to be matched can expand outward along the second direction.

[0089] Among them, the second side is the side that is parallel to the second direction and is located in the second direction of the image block to be matched.

[0090] Among them, the method for determining whether the new image to be matched can expand outward can be to determine whether the edge pixel points of the new image to be matched in the second direction coincide with the edge pixel points of the image to be processed in the second direction. If they coincide, it is determined that it cannot expand outward; if they do not coincide, it is determined that it can expand outward. It can also be determined whether it can expand outward according to whether the edge pixel points in the second direction coincide with the maximum boundary in the second direction. Among them, the maximum boundary can determine whether it can continue to expand outward according to the moving rule of the sliding window and the size of the sliding window. Among them, if they do not coincide, it can continue to expand outward; if they coincide, it cannot continue to expand outward. For the specific description of the maximum boundary in the second direction, reference can be made to the description of step S1402 in the foregoing embodiment, which will not be elaborated here one by one.

[0091] If it is determined that it can expand outward, then step S1413 is executed: expand the second edge of the image block to be matched in the second direction by N pixels to obtain a second candidate image block, where the second candidate image block includes the image block to be matched, and N is a positive integer.

[0092] Step S1414: Determine a second reference feature value according to the integral image of the target historical image and the position of the second candidate image block in the target historical image.

[0093] Step S1414: Obtain a second actual feature value of the second candidate image block according to the pixel parameters of each pixel in the second candidate image block.

[0094] Step S1415: Determine whether the second reference feature value matches the second actual feature value.

[0095] For the specific implementation principle of the above steps S1411 - S1415, reference can be made to the specific processes of steps S1401 - S1406 in the foregoing embodiment, which will not be elaborated here one by one.

[0096] If the second actual feature value matches the second reference feature value, then step S1416 is executed: determine the second candidate image block as a new image block to be matched, and return to step S1412.

[0097] If it is determined that it cannot expand outward or the second actual feature value does not match the second reference feature value, then step S1417 is executed: delete the second direction from the initial direction group, and step S1418 is executed: determine whether there is a direction in the initial direction group. If it is determined that there is a direction in the initial direction group, then return to step S1411 until the second actual feature value does not match the second reference feature value and there is no direction in the initial direction group, then step S1419 is executed: use the last determined new image block to be matched as the target image block.

[0098] By adopting the above steps S1411 - S1419, it is determined whether to continue expanding outwards by checking whether the edge pixel points of the new image block to be matched in the second direction coincide with the image to be processed or the maximum boundary. And since each expansion is accompanied by the comparison of eigenvalues, areas that do not meet the conditions can be quickly excluded, thus accelerating the entire matching process and quickly finding the maximum matching image area between the image to be processed and the target historical image.

[0099] In an implementable manner, refer to Figure 7 , the method further includes:

[0100] If there is no pre - stored eigenvalue that matches the eigenvalue of the image block to be matched, then execute step S150: control the sliding window to slide according to a preset movement rule. And return to execute the step of intercepting the image block to be matched from the image to be processed by using the sliding window.

[0101] Among them, the specific description of the preset movement rule can refer to the specific description of the foregoing step S140.

[0102] Considering that during the process of a user viewing an image, the user usually only slides the displayed image up and down and rarely drags the image left and right. Based on this, in an implementable manner of the present application, the preset movement rule is used to instruct the sliding window to jump - move at preset pixel intervals from left to right along the image to be processed, and when the right - hand side edge pixel point of the image block to be matched intercepted by the sliding window is the right - hand side edge pixel point of the image to be processed, move the sliding window down one pixel point along the image to be processed, and control the left - hand side edge of the sliding window to move to the left - hand side edge pixel point of the image to be processed, so as to perform jump - movement at preset pixel intervals from left to right along the image to be processed subsequently.

[0103] The number of the above - mentioned preset pixels is less than the number of pixels arranged in the width direction of the sliding window.

[0104] By adopting the above settings, it can be ensured that the sliding window can traverse the entire image area without missing any possible matching positions. At the same time, considering that during the process of a user viewing an image, the user usually only slides the displayed image up and down and rarely drags it left and right. Therefore, adopting the above - mentioned preset movement rule can reduce a lot of useless matches, thus greatly accelerating the matching process; it avoids the need to exhaustively check each column of pixels by using the sliding window, but makes a reasonable assumption according to the actual usage situation (such as the user's preference for scrolling up and down rather than dragging left and right), thereby selecting the optimal search strategy, so as to significantly improve the efficiency while ensuring accuracy.

[0105] It is worth mentioning that if, after moving the sliding window to the last position and obtaining the image block to be matched using the above method, no pre-stored image block of the image to be processed that matches the image block to be matched is found, the sliding rule of the sliding window can be adjusted. The adjusted sliding rule is that the sliding window moves one pixel to the right along the image to be processed. When the right-side edge pixel of the image block to be matched intercepted by the sliding window reaches the right-side edge pixel of the image to be processed, the sliding window moves one pixel down along the image to be processed, and the left-side edge of the sliding window is controlled to move to the left-side edge pixel of the image to be processed, so as to perform a jump movement at a preset pixel interval from left to right along the image to be processed. After the adjustment of the sliding rule is completed, return to execute the foregoing step S110. If, after moving the sliding window to the last position and obtaining the image block to be matched using the foregoing method, no pre-stored image block of the image to be processed that matches the image block to be matched is found, it can be determined that there is no target image area in the image to be processed.

[0106] In an implementable manner, please refer to Figure 8 , the method further includes:

[0107] Step S160: Intercept the image area to be transmitted from the image to be processed, where the image area to be transmitted includes other image areas in the image to be processed except the target image block.

[0108] Step S170: Transmit the image area to be transmitted and the position information of the target image block in the target historical image to the target client.

[0109] By adopting the above settings, it is possible to transmit only other image areas (i.e., the image area to be transmitted) in the image to be processed except the target image block, and the position information of the target image block in the target historical image, rather than the entire image to be processed, which greatly reduces the amount of data to be transmitted. In addition, after receiving the above-mentioned image area to be transmitted and the position information of the target image block in the target historical image, the target client can obtain the target image block according to the position information of the target image block in the target historical image block, and restore the image to be processed based on the target image block and the image area to be transmitted for display.

[0110] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0111] Please refer to Figure 9 , another embodiment of the present application provides an image matching device 200, including a data acquisition module 210 for acquiring an image to be processed; an image block intercepting module 220 for intercepting an image block to be matched from the image to be processed by using a sliding window; a feature value matching module 230 for matching the feature value of the image block to be matched with a plurality of pre-stored feature values, each of the pre-stored feature values corresponding to a pre-stored image block, the pre-stored image block being obtained by dividing a historical image, and the shape and size of the pre-stored image block being the same as those of the sliding window; a region searching module 240 for, when there is a target pre-stored feature value that matches the feature value of the image block to be matched, determining a target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored feature value in the target historical image, the integral image of the target historical image, and the position of the image block to be matched in the image to be processed, the target image block being the largest matching image region between the image to be processed and the target historical image, and the target image block including the image block to be matched.

[0112] In an implementable manner, the image matching device 200 further includes a sliding control module for controlling the sliding window to slide according to a preset movement rule when there is no pre-stored feature value that matches the feature value of the image block to be matched.

[0113] In an implementable manner, the region searching module 240 is further configured to expand the image block to be matched in at least one of the up, down, left, and right directions of the image to be processed by using the integral image of the target historical image and the position of the image block to be matched corresponding to the target historical image to obtain a target image block.

[0114] In an implementable embodiment, the region search module 240 includes a first direction selection sub-module, an edge movement sub-module, a first judgment sub-module, a first reference feature value determination sub-module, a first actual feature value determination sub-module, a reference boundary determination sub-module, a first direction deletion sub-module, and a first determination sub-module. The first direction selection sub-module is configured to select one direction from an initial direction group as the first direction, where the initial direction group includes four directions: up, down, left, and right; the edge movement sub-module is configured to move a first edge of the image block to be matched to the maximum boundary in the first direction to obtain a first candidate image block, where the first edge is the edge parallel to and having the shortest distance from the maximum boundary in the first direction; the first judgment sub-module is configured to determine whether a dichotomy stop condition is satisfied; the first reference feature value determination sub-module is configured to, when the dichotomy stop condition is satisfied, determine a first reference feature value according to the integral image of the target historical image and the position of the first candidate image block in the target historical image; the first actual feature value determination sub-module is configured to obtain a first actual feature value of the first candidate image block according to the pixel parameters of each pixel in the first candidate image block; the reference boundary determination sub-module is configured to, according to the matching result between the first actual feature value and the first reference feature value, use dichotomy to determine a reference boundary based on the first edge of the image block to be matched in the image to be processed and the maximum boundary in the first direction, and use the reference boundary as the new maximum boundary; the first direction deletion sub-module is configured to, when the dichotomy stop condition is satisfied, use the first candidate image block as the new image block to be matched and delete the first direction from the initial direction group; the first determination sub-module is configured to, when the first actual feature value matches the first reference feature value and there is no direction in the initial direction group, use the first candidate image block corresponding to the first actual feature value as the target image block.

[0115] In an implementable embodiment, the preset movement rule is used to instruct the sliding window to perform a jump movement at a preset pixel interval from left to right along the image to be processed, and when the right side edge pixel point of the image block to be matched intercepted by the sliding window is the right side edge pixel point of the image to be processed, move the sliding window down one pixel point along the image to be processed, and control the left side edge of the sliding window to move to the left side edge pixel point of the image to be processed, so as to perform a jump movement at a preset pixel interval from left to right along the image to be processed subsequently.

[0116] In an implementable manner, when the first direction is upward, the distance between the maximum boundary in the first direction and the upper edge pixel points of the to-be-matched image block is the height of the sliding window; when the first direction is downward, the maximum boundary in the first direction is the lower edge of the to-be-processed image; when the first direction is leftward, the distance between the maximum boundary in the first direction and the left edge pixel points of the to-be-matched image block is the width of the sliding window; when the first direction is rightward, the maximum boundary in the first direction is the right edge of the to-be-processed image.

[0117] In an implementable manner, the region search module 240 further includes a second direction selection sub-module, an expansion determination sub-module, an expansion sub-module, a second reference feature value determination sub-module, a second actual feature value determination sub-module, an image block update module, a second direction deletion sub-module, and a second determination sub-module. The second direction selection sub-module is configured to select one direction from the initial direction group as the second direction, where the initial direction group includes four directions: up, down, left, and right; the expansion determination sub-module is configured to determine whether the second side of the to-be-matched image block can expand outward along the second direction, where the second side is the side parallel to the second direction and located in the second direction of the to-be-matched image; the expansion sub-module is configured to, when it is determined that the expansion is possible, expand the second edge of the to-be-matched image block along the second direction by N pixels to obtain a second candidate image block, where the second candidate image block includes the to-be-matched image block, and N is a positive integer; the second reference feature value determination sub-module is configured to determine a second reference feature value according to the integral image of the target historical image and the position of the second candidate image block in the target historical image; the second actual feature value determination sub-module is configured to obtain the second actual feature value of the second candidate image block according to the pixel parameters of each pixel in the second candidate image block; the image block update module is configured to, if the second actual feature value matches the second reference feature value, determine the second candidate image block as the new to-be-matched image block. The second direction deletion sub-module is configured to, when it is determined that the expansion is not possible or the second actual feature value does not match the second reference feature value, delete the second direction from the initial direction group. The second determination sub-module is configured to, when the second actual feature value does not match the second reference feature value and there is no direction in the initial direction group, use the last determined new to-be-matched image block as the target image block.

[0118] In an implementable manner, the feature value is a hash value, and the multiple pre-stored feature values are obtained by performing hash feature extraction on each of the multiple pre-stored image blocks obtained by dividing at least one historical image using the sliding window.

[0119] In an implementable manner, the image matching device 200 further includes a region intercepting module and a data sending module. The region intercepting module is configured to intercept a to-be-transmitted image region from the to-be-processed image, where the to-be-transmitted image region includes other image regions in the to-be-processed image except the target image block; the data sending module is configured to transmit the to-be-transmitted image region and the position information of the target image block in the target historical image to a target client.

[0120] Each module in the above device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in the form of hardware or be independent of it, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules. It should be noted that the device embodiments in this application correspond to the foregoing method embodiments. The specific principles in the device embodiments can be referred to the content in the foregoing method embodiments, and will not be elaborated here.

[0121] Next, Figure 10 an electronic device provided by this application will be described.

[0122] Please refer to Figure 10 , based on the image matching method provided in the foregoing embodiment, another electronic device 300 provided in an embodiment of this application includes a processor 310 that can execute the foregoing method. The electronic device 300 can be a server, a terminal device, or a vehicle. The terminal device can be a device such as a smart phone, a tablet computer, a computer, or a portable computer.

[0123] The electronic device 300 further includes a memory 320. Among them, a program that can execute the content in the foregoing embodiment is stored in the memory 320, and the processor 310 can execute the program stored in the memory 320.

[0124] Among them, the processor 310 may include one or more cores for processing data and a message matrix unit. The processor 310 is connected to various parts within the entire electronic device 300 through various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 320, and by calling the data stored in the memory 320, it performs various functions of the electronic device 300 and processes data. Optionally, the processor 310 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 310 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the displayed content; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 310 and may be implemented separately through a communication chip.

[0125] The memory 320 may include random access memory (RAM) and may also include read-only memory. The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function, instructions for implementing the following various method embodiments, etc. The data storage area may also store data obtained during the use of the electronic device 300 (such as application information or skin images), etc.

[0126] The electronic device 300 may further include a network module and a screen. The network module is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with an audio playback device. The network module may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity module (SIM) cards, memories, and the like. The network module can communicate with various networks such as the Internet, enterprise intranets, wireless networks or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular phone networks, wireless local area networks or metropolitan area networks. The screen can display interface content and perform data interaction, such as displaying the aforementioned interface and triggering operations through the screen, etc.

[0127] An embodiment of the present application also provides a computer-readable storage medium. Program code is stored in the computer-readable medium, and the program code can be called by a processor to execute the method described in the above method embodiment.

[0128] The computer-readable storage medium may be an electronic memory such as a flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has a storage space for program code for executing any method step in the above method. These program codes can be read from one or more computer program products or written into these one or more computer program products. The program code can be compressed in an appropriate form, for example.

[0129] An embodiment of the present application also provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods described in the above various optional implementation manners.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image matching method, characterized in that: The method comprises: Get the image to be processed; Using a sliding window to intercept the image block to be matched from the image to be processed; Matching the feature value of the image block to be matched with a plurality of pre-stored feature values, each of the pre-stored feature values ​​corresponds to a pre-stored image block, the pre-stored image block is obtained by dividing a historical image, and the shape and size of the pre-stored image block are the same as the shape and size of the sliding window; If there is a target pre-stored feature value that matches the feature value of the image block to be matched, a target image block is determined from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored feature value in the target historical image, the integral image of the target historical image, and the position of the image block to be matched in the image to be processed, the target image block being the maximum matching image area between the image to be processed and the target historical image, and the target image block includes the image block to be matched.

2. The method according to claim 1, characterized in that The method further comprises: If there is no pre-stored feature value matching the feature value of the image block to be matched, the sliding window is controlled to slide according to a preset movement rule, and the step of using the sliding window to intercept the image block to be matched from the image to be processed is returned to be executed.

3. The method according to claim 2, characterized in that The step of determining a target image block from the image to be processed according to a position of a pre-stored image block corresponding to the target pre-stored feature value in the target historical image, an integral image of the target historical image, and a position of the image block to be matched in the image to be processed comprises: By using the integral image of the target historical image and the position of the image block to be matched corresponding to the target historical image, the image block to be matched is expanded along at least one of the four directions of up, down, left, and right of the image to be processed to obtain the target image block.

4. The method according to claim 3, characterized in that The method of using the integral image of the target historical image and the position of the image block to be matched corresponding to the target historical image to expand the image block to be matched along at least one of the four directions of up, down, left, and right of the image to be processed to obtain the target image block comprises: Selecting a direction from an initial direction group as a first direction, wherein the initial direction group includes four directions: up, down, left, and right; Moving a first side of the image block to be matched to a maximum boundary in a first direction to obtain a first candidate image block, wherein the first side is a side parallel to the maximum boundary in the first direction and with the shortest distance; Determine whether the dichotomy stopping condition is met; If not satisfied, determining a first reference feature value according to an integral image of a target historical image and a position of the first candidate image block in the target historical image; Obtaining a first actual feature value of the first candidate image block according to a pixel parameter of each pixel in the first candidate image block; According to the matching result between the first actual eigenvalue and the first reference eigenvalue, a reference boundary is determined by using a binary search method based on the maximum boundary of the image block to be matched on the first side of the image to be processed and in the first direction; The reference boundary is used as a new maximum boundary, and the step of moving the first side of the image block to be matched to the maximum boundary in the first direction to obtain a first candidate image block is returned to execute until the dichotomy stop condition is met, the first candidate image block is used as a new image block to be matched, the first direction is deleted from the initial direction group, and the step of selecting a direction from the initial direction group in which the first direction is the image to be processed as the first direction is returned to execute until the first actual eigenvalue matches the first reference eigenvalue, and when there is no direction in the initial direction group, the first candidate image block corresponding to the first actual eigenvalue is used as the target image block.

5. The method according to claim 4, characterized in that The preset movement rule is used to instruct the sliding window to jump and move from left to right along the image to be processed at intervals of preset pixels, and when the right side edge pixel point of the image block to be matched captured by the sliding window is the right side edge pixel point of the image to be processed, the sliding window is moved one pixel point from top to bottom along the image to be processed, and the left side edge of the sliding window is controlled to move to the left side edge pixel point of the image to be processed, so as to jump and move from left to right along the image to be processed at intervals of preset pixels subsequently.

6. The method according to claim 5, characterized in that When the first direction is upward, the distance between the maximum boundary of the first direction and the upper edge pixel point of the image block to be matched is the height of the sliding window; when the first direction is downward, the maximum boundary of the first direction is the lower edge of the image to be processed; when the first direction is left, the distance between the maximum boundary of the first direction and the left edge pixel point of the image block to be matched is the width of the sliding window; when the first direction is right, the maximum boundary of the first direction is the right edge of the image to be processed.

7. The method according to claim 3, characterized in that The method of using the integral image of the target historical image and the position of the image block to be matched corresponding to the target historical image to expand the image block to be matched along at least one of the four directions of up, down, left, and right of the image to be processed to obtain the target image block comprises: Selecting a direction from an initial direction group as a second direction, wherein the initial direction group includes four directions: up, down, left, and right; Determine whether a second side of the image block to be matched can be expanded outward along a second direction, wherein the second side is a side parallel to the second direction and located in the second direction of the image block to be matched; If it is determined that the image block can be expanded outward, the second edge of the image block to be matched is expanded by N pixels along the second direction to obtain a second candidate image block, where the second candidate image block includes the image block to be matched, wherein N is a positive integer; Determine a second reference feature value according to the integral image of the target historical image and the position of the second candidate image block in the target historical image; Obtaining a second actual feature value of the second candidate image block according to a pixel parameter of each pixel in the second candidate image block; If the second actual eigenvalue matches the second reference eigenvalue, the second candidate image block is determined as a new image block to be matched, and the step of determining whether the second side of the image block to be matched can be expanded outward along the second direction is returned to execute; until it is determined that it cannot be expanded outward or the second actual eigenvalue does not match the second reference eigenvalue, the second direction is deleted from the initial direction group, and the step of selecting a direction from the initial direction group as the second direction is returned to execute, until the second actual eigenvalue does not match the second reference eigenvalue and there is no direction in the initial direction group, the new image block to be matched determined for the last time is used as the target image block.

8. The method according to claim 1, characterized in that: The feature value is a hash value, and the multiple pre-stored feature values ​​are obtained by dividing at least one historical image using the sliding window to obtain multiple pre-stored image blocks, and then performing hash feature extraction on each of the pre-stored image blocks.

9. The method according to claim 1, characterized in that: The method further comprises: Cutting out an image region to be transmitted from the image to be processed, wherein the image region to be transmitted includes other image regions in the image to be processed except the target image block; Transmitting the image area to be transmitted and the position information of the target image block in the target historical image to the target client.

10. An image matching device, characterized in that: The device comprises: A data acquisition module, used for acquiring an image to be processed; An image block interception module, used for intercepting the image block to be matched from the image to be processed by using a sliding window; A feature value matching module, used for matching the feature value of the image block to be matched with a plurality of pre-stored feature values, each of the pre-stored feature values ​​corresponds to a pre-stored image block, the pre-stored image block is obtained by dividing the historical image, and the shape and size of the pre-stored image block are the same as the shape and size of the sliding window; A region search module is used to determine a target image block from the image to be processed according to the position of the pre-stored image block corresponding to the target pre-stored feature value in the target historical image, the integral image of the target historical image, and the position of the image block to be matched in the image to be processed when there is a target pre-stored feature value that matches the feature value of the image block to be matched, wherein the target image block is the maximum matching image region between the image to be processed and the target historical image, and the target image block includes the image block to be matched.

11. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images