Image processing method, device, storage medium and computer equipment

Abstract

The present application discloses an image processing method, apparatus, storage medium and computer equipment; the present application is related to the field of machine learning and computer vision of artificial intelligence, and can obtain at least one frame of image, perform element recognition on at least one frame of image, and obtain the position information and category information of at least one element in each frame of image; determine at least one target element set according to the category information of the elements in all images, the target element set includes at least one group of target element pairs with the same target category information, and the target element pair includes a first target element and a second target element; calculate the position range information of the first target element in the second image, and the second image is the image where the second target element is located; based on the position range information and the position information of the second target element, perform credibility analysis on the target category information of the target element pair; the present application can improve the accuracy of image processing, and can be applied in fields including but not limited to maps, navigation, smart transportation, automatic driving, etc.

Description

Technical Field

[0001] The present application relates to the field of images, and in particular to an image processing method, apparatus, storage medium and computer equipment. Background Art

[0002] Image recognition technology can be used to identify road element information, such as traffic facilities (speed limit signs, signal lights, traffic signs, etc.), buildings (houses, buildings, etc.), municipal facilities (electronic eyes, street lights, etc.), and other element category information. For example, you can take a road image and detect it through a category detection model to obtain the element category information of the elements on the road.

[0003] However, the existing recognition of road elements has problems such as low recognition accuracy, especially when affected by factors such as image shooting scene and weather conditions. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, storage medium and computer equipment, which can effectively improve the accuracy of image element recognition.

[0005] The present application provides an image processing method, including:

[0006] Acquire at least one frame of image, perform element recognition on the at least one frame of image, and obtain position information and category information of at least one element in each frame of image;

[0007] Determine at least one target element set according to category information of elements in all images, wherein the target element set includes at least one group of target element pairs having the same target category information, wherein the target element pairs include a first target element and a second target element;

[0008] Calculating position range information of the first target element in a second image, where the second image is an image where the second target element is located;

[0009] Based on the position range information and the position information of the second target element, a credibility analysis is performed on the target category information of the target element pair.

[0010] Accordingly, the present application provides an image processing device, comprising:

[0011] An element recognition module is used to obtain at least one frame of image, perform element recognition on the at least one frame of image, and obtain position information and category information of at least one element in each frame of image;

[0012] A determination module, configured to determine at least one target element set according to category information of elements in all images, wherein the target element set includes at least one group of target element pairs having the same target category information, wherein the target element pairs include a first target element and a second target element;

[0013] A calculation module, used for calculating the position range information of the first target element in a second image, where the second image is an image where the second target element is located;

[0014] The credibility analysis module is used to perform credibility analysis on the target category information of the target element pair based on the position range information and the position information of the second target element.

[0015] In some embodiments, the credibility analysis module includes a calculation submodule and a determination submodule, wherein:

[0016] a calculation submodule, configured to calculate the initial credibility of the target category information of the target element pair based on the position range information and the position information of the second target element;

[0017] A determination submodule is used to determine the target credibility of the target category information of the second target element according to the initial credibility.

[0018] In some embodiments, the calculation submodule includes a calculation unit and a comparison unit, wherein:

[0019] a calculation unit, configured to calculate distance confidence information of the second target element based on the position range information and the position information of the second target element;

[0020] A comparison unit is used to compare the distance confidence information with a preset confidence threshold to determine the initial credibility of the target category information of the target element pair.

[0021] In some embodiments, the determination submodule is specifically used for:

[0022] Determine at least one candidate credibility of the target category information of the second target element, the candidate credibility includes credibility of other target element pairs to which the second target element belongs, the other target element pairs being target element pairs in the target element set;

[0023] Based on the initial credibility and the candidate credibility, the target credibility of the target category information of the second target element is determined.

[0024] In some embodiments, the calculation module includes a calculation submodule, a determination submodule and a fusion submodule, wherein:

[0025] a calculation submodule, used to calculate a first stereoscopic optical center when a first element image is taken, and to calculate a second stereoscopic optical center when a second element image is taken, wherein the first element image is an image where the first target element is located;

[0026] A determination submodule, used to determine the three-dimensional coordinates of the first target element in the real environment;

[0027] A fusion submodule is used to fuse the first stereo optical center, the second stereo optical center and the element stereo coordinates to obtain the position range information of the first target element in the second image.

[0028] In some embodiments, the determination submodule is specifically used for:

[0029] Acquire a first shooting position and a first shooting parameter when the first image is shot;

[0030] The first shooting position, the first shooting parameter and the position information of the first target element are fused to obtain the element stereo coordinates of the first target element in the real environment.

[0031] In some embodiments, the computing submodule is specifically used for:

[0032] Acquire a first shooting position and a first shooting parameter when the first image is shot, and acquire a second shooting position and a second shooting parameter when the second image is shot;

[0033] The first stereoscopic optical center when the first element image is photographed is calculated according to the first photographing position and the first photographing parameters, and the second stereoscopic optical center when the second image is photographed is calculated according to the second photographing position and the second photographing parameters.

[0034] In some embodiments, the element detection module is specifically used to:

[0035] Performing feature extraction on the at least one frame of image to obtain image features of each frame of image, wherein the image features include at least one feature point;

[0036] Determine multiple candidate element regions with the feature point as the center in each frame of the image;

[0037] Element detection and classification are performed on the candidate element regions, and a target element region, position information of the target element region, and category information of elements in the target element region are determined from the candidate element regions, thereby obtaining position information and category information of at least one element in each frame image.

[0038] In some embodiments, the element detection module includes a module specifically configured to:

[0039] The trained element recognition model is used to perform element recognition on the at least one frame of image to obtain location information and category information of at least one element in each frame of image.

[0040] In some embodiments, the image processing apparatus further comprises:

[0041] A sample acquisition module, used to acquire a plurality of sample images and sample labels corresponding to the sample images;

[0042] A construction module for constructing a feature recognition model;

[0043] The training module is used to train the element recognition model based on the sample image and the sample label to obtain a trained element recognition model.

[0044] Correspondingly, an embodiment of the present application further provides a storage medium, which stores a computer program, and the computer program is suitable for loading by a processor to execute any image processing method provided in the embodiment of the present application.

[0045] Correspondingly, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the image processing methods provided in the embodiment of the present application is implemented.

[0046] The present application can acquire at least one frame of image, perform element recognition on at least one frame of image, and obtain location information and category information of at least one element in each frame of image; determine at least one target element set based on the category information of the elements in all images, the target element set includes at least one group of target element pairs with the same target category information, the target element pair includes a first target element and a second target element; calculate the location range information of the first target element in the second image, the second image is the image where the second target element is located; based on the location range information and the location information of the second target element, perform credibility analysis on the target category information of the target element pair.

[0047] After detecting the position information and category information of the elements, the present application can determine the pair of target elements with the same target category information, and then determine the position range information of the first target element in the second image where the second target element is located, and then based on the position information of the second target element and this position range information, perform credibility analysis on the target position information of the first target element and the second target element, so as to further determine whether the target category information is accurate based on the credibility analysis result, thereby improving the accuracy of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 is a schematic diagram of a scene of an image processing system provided in an embodiment of the present application;

[0050] Figure 2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of the implementation principle of the image processing method provided in the embodiment of the present application;

[0052] Figure 4 is another schematic diagram of the implementation principle of the image processing method provided in the embodiment of the present application;

[0053] Figure 5 is another schematic diagram of the implementation principle of the image processing method provided in the embodiment of the present application;

[0054] Figure 6 is another flowchart of the image processing method provided by an embodiment of the present application;

[0055] Figure 7 It is a schematic diagram of a partial implementation of the image processing method provided in the embodiment of the present application;

[0056] Figure 8 is an example diagram of the overall process of the image processing method provided in the embodiment of the present application;

[0057] Fig. 9 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0058] Fig.10 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described in this application are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0060] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0061] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, detect and measure targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0062] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0063] The image processing method of the present application is related to the technologies in the field of machine learning and computer vision of artificial intelligence. For example, the image can be detected through a convolutional neural network model to obtain the position information and category information of at least one element in the image, etc., which will be discussed in detail in the embodiments.

[0064] The image processing method of the present application can be integrated into an image processing system, which can be integrated into one or more computer devices, which can include a terminal or a server, etc., wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart home, a wearable electronic device, a vehicle-mounted computer, a vehicle-mounted smart terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0065] See also Figure 1 The image processing system may include an image processing device, wherein the image processing device can acquire at least one frame of image, perform element recognition on at least one frame of image, and obtain position information and category information of at least one element in each frame of image; determine at least one target element set according to the category information of elements in all images, the target element set includes at least one group of target element pairs with the same target category information, and the target element pair includes a first target element and a second target element; calculate the position range information of the first target element in the second image, the second image is the image where the second target element is located; based on the position range information and the position information of the second target element, perform credibility analysis on the target category information of the target element pair.

[0066] It should be noted that Figure 1 The scene diagram of the image processing system shown is only an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of image processing devices and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0067] In this embodiment, the image processing method is described in detail. The image processing method can be integrated in a terminal or a server, or can be implemented by a terminal and a server together. Figure 2 As shown, Figure 2 : is a flow chart of an image processing method provided in an embodiment of the present application. The image processing method may include:

[0068] 101. Acquire at least one frame of image, perform element recognition on the at least one frame of image, and obtain position information and category information of at least one element in each frame of image.

[0069] The image may include an image acquired for feature recognition. There are many ways to acquire the image. The image may be acquired from a local computer device or other computer devices, or may be acquired by photographing the image with a camera device, etc.

[0070] In one or more embodiments, a data acquisition request may be sent to a server, and at least one frame of image returned by the server based on the data acquisition request may be received.

[0071] In one or more embodiments, the at least one frame of image may include at least two frames of image, at least three frames of image, and so on.

[0072] Among them, elements may include specific content on the image. According to the needs of the actual application scenarios, the elements can be flexibly changed. For example, elements may include express delivery orders, traffic signs, fruits, plants, commodities, animals, transportation facilities, buildings, municipal facilities, etc.

[0073] The position information may include the position of the element obtained by recognition (i.e., detection) in the image in which it is located. The position information may be identified based on the coordinate information of the element. For example, if a square detection frame is obtained by element detection, the upper left coordinate and the upper right coordinate of the square detection frame may be recorded as the position information of the element. For another example, if a rectangular detection frame is obtained by element recognition, the upper right coordinate of the rectangular frame and the length and width of the rectangular frame may be recorded as the position information of the element.

[0074] The category information may include the subdivision of elements. For example, traffic facilities may include speed limit signs, no-entry signs, U-turn signs, traffic lights, etc. For another example, buildings may include Building A, School B, etc.

[0075] In one or more embodiments, the process of element recognition (i.e., element detection) of an image may include multiple methods. In one or more embodiments, element detection may be performed based on a related model (such as a YOLO V1 model, a YOLO V3 model, a YOLO V4 model, etc.) under a YOLO framework (an image detection algorithm framework), or by using an R-FCN (Region-based fully convolutional network) model (an element detection model), an SSD model (an element detection model), etc.

[0076] Feature detection can also be performed based on a convolutional neural network (CNN) algorithm. The convolutional neural network can be trained using sample data before application. After the training is completed, the trained convolutional neural network can be used for feature detection. Specifically, it can include RCNN (a feature detection algorithm), SPPNet (a feature detection algorithm), Fast R-CNN (a feature detection algorithm), Faster R-CNN (a feature detection algorithm), etc.

[0077] The process of element detection can be flexibly selected according to the actual application scenario and the characteristics of the target elements, which will not be elaborated here.

[0078] For example, element recognition is performed on multiple frames of images respectively to obtain the position information yw1 and category information of element y1 in image 1:, the position information yw2 and category information of element y2: no passage sign, and the position information yw3 and category information of element y3 in image 2: no right turn sign.

[0079] In some embodiments, the step of “performing element recognition on at least one frame of image to obtain location information and category information of at least one element in each frame of image” may include:

[0080] Extracting features from at least one frame of image to obtain image features of each frame of image, where the image features include at least one feature point;

[0081] In each frame of the image, multiple candidate feature regions are determined with the feature point as the center;

[0082] Detect and classify the elements in the candidate element regions, determine the target element region, the location information of the target element region, and the category information of the elements in the target element region from the candidate element regions, so as to obtain the location information and category information of at least one element in each frame image.

[0083] In one or more embodiments, during feature recognition (i.e., feature detection), feature extraction can be performed through a feature extraction layer, which may include a convolution layer, a normalization layer BatchNormalization (BN), and an activation layer Relu. The convolution layer can extract basic features such as edge texture of the image. The normalization layer can normalize the features extracted by the convolution layer according to a normal distribution, filter out noise features in the basic features, and make the feature extraction layer training converge faster. The activation layer can perform nonlinear mapping on the features extracted by the convolution layer, and enhance the generalization ability of the feature extraction layer.

[0084] Feature extraction can obtain image features of an image. Image features can include multiple feature points. Then, the candidate feature area corresponding to the feature point can be determined based on different aspect ratios and the number of feature points, with the feature point in the image as the center. The aspect ratio is the aspect ratio of the candidate feature area obtained with the feature point as the center, and the number of feature points is the number of feature points on the long side or the wide side of the candidate feature area. For example, the aspect ratio can include 1:1, 1:2 and 2:1, and the number of feature points can include 1, 2 and 3. See Figure 3 For feature point O, the above aspect ratio and the number of feature points can be combined to obtain 9 candidate feature regions. Then, the target feature region, as well as the category information and location information of the target feature region, can be determined from the candidate feature regions by a classifier.

[0085] In some embodiments, the step of “performing element recognition on at least one frame of image to obtain location information and category information of at least one element in each frame of image” may include:

[0086] The trained feature recognition model is used to perform feature recognition on at least one frame of image to obtain location information and category information of at least one feature in each frame of image.

[0087] For example, the trained feature recognition model can be found in Figure 4 The feature recognition model may include a feature extraction layer, a candidate box selection network, a classifier and other parts. Through operations such as feature extraction, candidate box selection, pooling dimensionality reduction, and classification, at least one feature of the input image, as well as the location information and category information of the feature, can be obtained.

[0088] In some embodiments, the image processing method further comprises:

[0089] Acquire multiple sample images and sample labels corresponding to the sample images; construct a feature recognition model; train the feature recognition model based on the sample images and the sample labels to obtain a trained feature recognition model.

[0090] Specifically, the feature recognition model can be trained based on sample images before performing feature recognition. The model training can adjust the model parameters of the feature recognition model based on the loss value obtained by the model output and the sample label. Specifically, the algorithms referred to for adjusting the model parameters may include multiple algorithms, such as stochastic gradient descent (SGD), stochastic gradient descent using momentum (Momentum SGD), adaptive gradient (AdaGrad), etc. The selection of the algorithm can be flexibly determined in the actual application process, and will not be repeated here.

[0091] 102. Determine at least one target element set according to category information of elements in all images, where the target element set includes at least one group of target element pairs with the same target category information, and the target element pairs include a first target element and a second target element.

[0092] The target element set may include multiple target elements, and the category information of each target element in the target element set is the same. By combining the target elements in the target element set in pairs, a target element pair can be obtained.

[0093] In one or more embodiments, all identified elements can be classified according to the result of element recognition, and elements with the same category information can be integrated into a target element set. Then, the category information of the target elements in the target element set can be analyzed for credibility, so as to obtain more accurate element detection results. In a target element set, at least two target elements with the same category information can be included, and these target elements can be combined, and each target element can be combined with other target elements to obtain at least one group of target element pairs in a target element set.

[0094] For example, target element set 1 may include 5 target elements with category information of "no entry sign", namely target element 1, target element 2, target element 3, target element 4 and target element 5, which can then be combined to obtain 10 groups of target element pairs, among which one group of target element pairs includes target element 1 and target element 3.

[0095] 103. Calculate position range information of the first target element in the second image, where the second image is an image where the second target element is located.

[0096] The position range information may include the position range where an element in one image may appear in another image. The position range information may be expressed in various forms, such as a coordinate range, an equation, and the like.

[0097] Specifically, there may be multiple ways to calculate the position range information of the first target element in the second image. For example, the position range information may be calculated based on the information of the first target element in the real environment and the information of the second image. For another example, the calculation may be based on the polar line constraint principle, and the obtained polar line may be the position range information, and so on.

[0098] For example, target element 1 belongs to image 1, and target element 3 belongs to image 3, and the possible position range information 1 of target element 1 in image 3 can be calculated.

[0099] In some embodiments, the step of “calculating the position range information of the first target element in the second image” may include:

[0100] Calculating a first stereoscopic optical center when a first image is taken, and calculating a second stereoscopic optical center when a second image is taken, wherein the first image is an image where the first target element is located;

[0101] Determine the element stereo coordinates of the first target element in the real environment;

[0102] The first stereo optical center, the second stereo optical center and the element stereo coordinates are fused to obtain the position range information of the first target element in the second image.

[0103] The position range information of the first target element in the second image can be determined according to the epipolar constraint principle. For details, see Figure 5 , where P is the position of the element in the real environment, the imaging position of P in the phase plane 1 of the shooting device 1 is P0, C0 is the optical center of the shooting device at this time, the imaging position of P in the phase plane 2 of the shooting device 1 is P1, C1 is the optical center of the shooting device 1 at this time, where phase plane 1 and phase plane 2 are different imaging planes of the shooting device during the mobile shooting process. The epipolar principle means that due to the constant internal parameters of the shooting device, when P0 of phase plane 1 and P in the real environment are known, it can be determined that the imaging position P1 of P in phase plane 2 must be on the epipolar line of the phase plane. Based on this principle, the epipolar line on phase plane 2 can be calculated based on the optical center of phase plane 1, the optical center of phase plane 2 and the real position of the element. The epipolar line is the possible imaging position range of point P0 on phase plane 1 on phase plane 2.

[0104] Among them, the first stereo optical center may include the three-dimensional optical center of the shooting device when shooting the first image where the first target element is located, the second stereo optical center may include the three-dimensional optical center of the shooting device when shooting the second image where the second target element is located, and the element stereo coordinates may include the three-dimensional coordinates of the entity corresponding to the first target element in the real environment.

[0105] There are many ways to obtain the first stereoscopic optical center and the second stereoscopic optical center. For example, a positioning device can be installed on the shooting device, and through relevant algorithms, the shooting device can automatically locate the stereoscopic optical center when shooting (such as pressing the shutter), etc.

[0106] The acquisition of the three-dimensional coordinates of the elements may include a variety of methods, for example, manually using a collection device to collect in the real environment where the real object corresponding to the first target element is located, or estimating based on relevant information related to the first target element, and so on.

[0107] Then, a fusion process can be performed based on the first stereo optical center, the second stereo optical center and the element stereo coordinates to obtain the epipolar line of the first target element in the second image. For example, the first stereo optical center, the second stereo optical center and the element stereo coordinates can be fused to determine the plane where these three positions are located, and then the intersection line of this plane with the phase plane when the second image is taken is calculated to obtain the epipolar line.

[0108] In some embodiments, the step of “determining the element stereo coordinates of the first target element in the real environment” may include:

[0109] A first shooting position and a first shooting parameter when the first image is shot are obtained; the first shooting position, the first shooting parameter and the position information of the first target element are fused to obtain the element stereo coordinates of the first target element in the real environment.

[0110] The element stereo coordinates of the object of the first target element in the real environment can be calculated. For example, the calculation can be performed based on the first shooting position when shooting the first image and the first shooting parameter of the shooting device, wherein the first shooting position may include the position of the shooting device in the real environment when shooting the first image, and the first shooting parameter may include the relevant shooting parameters set by the shooting device when shooting the first image, and at least one of the internal parameters of the shooting device. The fusion processing process can be flexibly performed according to actual needs. For example, the model can be modeled based on the first shooting parameter and the first shooting position parameters to obtain the element stereo coordinates. For example, the first shooting parameter and the first shooting position can be fused and calculated. The specific selection can be flexibly made when applying, and will not be repeated here.

[0111] For example, the shooting position 1 and shooting parameter 1 when the image 1 is shot may be obtained, and the shooting position 1 and the shooting parameter 1 may be fused and calculated to obtain the three-dimensional coordinates of the target element 1 in the real environment.

[0112] In some embodiments, the step of “calculating a first stereoscopic optical center when the first image is captured, and calculating a second stereoscopic optical center when the second image is captured” may include:

[0113] Acquire a first shooting position and a first shooting parameter when the first image is shot, and acquire a second shooting position and a second shooting parameter when the second image is shot; calculate a first stereoscopic optical center when the first image is shot according to the first shooting position and the first shooting parameter, and calculate a second stereoscopic optical center when the second image is shot according to the second shooting position and the second shooting parameter.

[0114] For example, the shooting position 1 and shooting parameter 1 when image 1 is shot can be obtained, and calculations can be performed based on the shooting position 1 and shooting parameter 1 to obtain the stereoscopic optical center 1 of the shooting device when shooting image 1; the shooting position 2 and shooting parameter 2 when image 2 is shot can be obtained, and calculations can be performed based on the shooting position 2 and shooting parameter 2 to obtain the stereoscopic optical center 2 of the shooting device when shooting image 2.

[0115] 104. Based on the location range information and the location information of the second target element, a credibility analysis is performed on the target category information of the target element pair.

[0116] After obtaining the position range information of the first target element in the second image, the present application can perform a credibility analysis on the position information of the target element pair based on the position range information and the position information of the second target element. If the position information of the second target element is within an acceptable range determined based on the position range information, it can be determined that the credibility of the category information of the target element pair is relatively high, that is, the credibility of the category information of the first target element and the second target element in the target element pair is relatively high; if not, it is determined that the credibility of the category information of the target element pair is relatively low, that is, the credibility of the category information of the first target element and the second target element in the target pixel pair is relatively low.

[0117] According to the credibility of the category information of all target element pairs in a target element set, at least one credibility of the category information of each target element in the target element set in at least one target element pair can be obtained, thereby determining the target credibility of the category information of each target element. The category information of the target elements with higher target credibility can be accepted, while the category information of the target elements with lower target credibility can be rejected. That is, the detected category information can be analyzed again to further screen out possible erroneous detection results, thereby improving the accuracy of image processing.

[0118] For example, the credibility of the category information “no entry sign” of the target element pair (target element 1 and target element 3) may be checked based on the position range information 1 in image 3 and the position information of the target element 3 in image 3.

[0119] In some embodiments, the step of “performing a credibility analysis on the target category information of the target element pair based on the location range information and the location information of the second target element” may include:

[0120] Based on the position range information and the position information of the second target element, the initial credibility of the target category information of the target element pair is calculated; and according to the initial credibility, the target credibility of the target category information of the second target element is determined.

[0121] The initial credibility may be content obtained by performing credibility analysis on the category information of the target element pair, and the target credibility may include the credibility of the category information of the target element pair.

[0122] The initial credibility of the target element pair can be calculated based on the position range information of the first target element in the second image and the position information of the second target element detected in the second image with the same category information as the first target element. The calculation method may include multiple methods. For example, if the position range information is an irregular line, such as a fold, the distance between the position information and each line can be calculated. If the distance between the position information and a line in the fold is zero, it is determined that the second target element is within the position range information, and the initial credibility of the target element pair is high. Otherwise, it is considered that the initial credibility of the target element pair is low.

[0123] Then, based on the initial credibility, the target credibility of the first target element and the second target element in the target element pair can be determined. Similarly, the target credibility of each target element pair in the target element set can be determined by the above method.

[0124] In some embodiments, the step of “calculating the initial credibility of the target category information of the target element pair based on the location range information and the location information of the second target element” may include:

[0125] Based on the position range information and the position information of the second target element, the distance confidence information of the second target element is calculated; and the size relationship between the distance confidence information and the preset confidence threshold is compared to determine the initial credibility of the target category information of the target element pair.

[0126] Specifically, the location range information can be a polar line equation, and the distance between the polar line equation and the location information of the second target element, that is, the distance from the point to the straight line, can be calculated. If the distance is less than a preset threshold, it can be determined that the initial credibility is high. If the distance is greater than the preset threshold, it can be determined that the initial credibility is low.

[0127] In the present application, the credibility (including initial credibility and target credibility) can be named data, and the credibility only includes two values: high credibility and low credibility, which can be specifically identified by characters, such as using 0 to identify high credibility and 1 to identify low credibility.

[0128] Specifically, the distance confidence information between the epipolar line and the position information of the second target element can also be calculated. For example, L1 is the epipolar line equation, P j+1 is the location information of the second target element, s is the distance confidence information between the second target element and the polar equation,

[0129]

[0130] Among them, dis(L1,P j+1 ) is the polar line L1 and P j+1 The distance between.

[0131] Then, the distance confidence information can be compared with the preset threshold. For example, the preset threshold can be 0.5. When s>0.5, the initial confidence of the target feature pair can be considered to be 0, and when s<0.5, the initial confidence of the target feature pair can be considered to be 1.

[0132] In some embodiments, the step of “determining the target credibility of the target category information of the second target element according to the initial credibility” may include:

[0133] Determine at least one candidate credibility of the target category information of the second target element, the candidate credibility includes the credibility of other target element pairs to which the second target element belongs, and the other target element pairs are target element pairs in the target element set; based on the initial credibility and the candidate credibility, determine the target credibility of the target category information of the second target element.

[0134] Specifically, the target element set may include multiple target elements, and all target elements may be combined with each other to obtain multiple target element pairs, that is, each target element and other target elements in the target element set may be combined into a target element pair. In the embodiment of the present application, the category information of all target element pairs in the target element set will be subjected to credibility analysis to obtain the initial credibility of each target element pair.

[0135] When determining the target credibility of the target element in the target element set, the initial credibility of the category information of all target element pairs to which the target element belongs can be obtained. According to all the initial credibility, the target credibility of the category information of the target element is determined. There are many ways to determine it. The value with a larger number of initial credibility can be determined as the target credibility. For example, target element 3 belongs to 7 target element pairs, and the initial credibility values ​​of these 7 target element pairs are 4 a (representing low initial credibility) and 3 b (representing high initial credibility). Then the target credibility of target element 3 can be determined to be a. A quantity threshold of the credibility value can also be set. For example, when the number of initial credibility values ​​is greater than or equal to N, the target credibility of the target element is determined to be the value. For example, target element 1 belongs to 7 target element pairs, and the initial credibility values ​​of these 7 target element pairs are 6 a (representing low initial credibility) and 1 b (representing high initial credibility). Then the number of initial credibility values ​​a, 6, is greater than 5, and it can be known that the target credibility of target element 1 is a.

[0136] The candidate credibility may include the credibility obtained by performing credibility analysis on category information of other target elements to which the second target element belongs in the target element set.

[0137] For example, the initial credibility of the category information of a set of target element pairs to which target element 1 belongs is a, and the candidate credibility of the category information of the other four sets of target element pairs to which target element 1 belongs in target element set 1 can be a, a, a, and b. Based on these five initial credibility, the target credibility of target element 1 can be determined to be 1.

[0138] After detecting the position information and category information of the elements, the present application can determine the pair of target elements with the same target category information, and then determine the position range information of the first target element in the second image where the second target element is located, and then based on the position information of the second target element and this position range information, perform credibility analysis on the target position information of the first target element and the second target element, so as to further determine whether the target category information is accurate based on the credibility analysis result, thereby improving the accuracy of image processing.

[0139] The method described in the above embodiment is further described in detail below with examples.

[0140] This application can identify road signs in road images, see Figure 6 , Figure 6 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application, the image processing method may include:

[0141] 201. The computer device acquires multiple frames of images.

[0142] For example, the on-board computer equipment takes a picture of the road ahead to obtain a road image sequence.

[0143] 202. The computer device performs element recognition on each frame of the image to obtain position information and category information of at least one element in each frame of the image.

[0144] For example, a computer device can detect road signs on each road image in a road image sequence through a convolutional neural network to obtain the location information and category information of the road signs in each road image. If the road image sequence includes m images and n road signs are detected on each image, the category information of the road signs detected in any image can be expressed as {cls i} j ,i=1,2,…,n,j=1,2,…,m, the position information can be expressed as {x i ,y i ,w i ,h i} j ,i=1,2,…,n,j=1,2,…,m, where m is greater than 0, n is greater than or equal to 0, x i ,y i, is the coordinate of the upper left corner of the detected rectangular box containing the road sign, w i ,h i Represents the width and height of the rectangle.

[0145] For example, see Figure 7 The right picture can be obtained by performing road sign recognition on the road image through a convolutional neural network. The road image 1 in the right picture may include the detected rectangular frame 1, the type information of the road sign in the rectangular frame 1: the scenic area distance sign, and the location information of the rectangular frame 1 {3, 3, 3, 5}.

[0146] 203. The computer device determines at least one target element set according to category information of elements in each frame of the image, where the target element set includes at least one group of target element pairs with the same target category information, and the target element pairs include a first target element and a second target element.

[0147] For example, based on the category information of road signs obtained from all road image detections, the detected road signs are classified to obtain three road sign detection sets. Each road sign detection set includes multiple road signs with the same category information. Among them, road sign detection set 1 includes five road signs whose category information is all scenic area distance signs. These five road signs are combined with each other to obtain 10 groups of road sign pairs to be detected, among which road sign pair 6 to be detected includes road sign A and road sign D.

[0148] 204. The computer device calculates position range information of the first target element in the second image, where the second image is the image where the second target element is located.

[0149] Wherein, road sign A is a road sign detected on road image 1, and road sign D is a road sign detected on road image 2. Based on the principle of epipolar constraint, the epipolar line L1 of road sign A on road image 2 can be calculated.

[0150] 205. The computer device calculates the distance confidence information of the second target element based on the location range information and the location information of the second target element.

[0151] For example, the inverse value S of the vertical distance between the road sign D and the polar line L1 may be calculated based on the coordinates in the position information of the road sign D.

[0152] 206. The computer device compares the distance confidence information with a preset confidence threshold to determine the initial credibility of the target category information of the target element pair.

[0153] For example, comparing the size relationship between S and 0.5, if S is 0.7, it can be determined that S is greater than 0.5, and the category information of the road sign pair 6 to be detected: the initial credibility of the scenic area distance sign is: G (representing high credibility).

[0154] 207. The computer device determines at least one candidate credibility of the target category information of the second target element, where the candidate credibility includes credibility of other target element pairs to which the second target element belongs, and the other target element pairs are target element pairs in the target element set.

[0155] For example, determine that the candidate credibility of road sign A in the other four groups of road sign pairs to be detected are: G, G, D (representing low credibility), and D.

[0156] 208. The computer device determines the target credibility of the target category information of the second target element based on the initial credibility and the candidate credibility.

[0157] For example, the target credibility is determined by a rule that if all credibility values ​​are the same, then the target credibility is the credibility value; otherwise, if the target credibility value is D, then the target credibility of road sign A can be determined to be D.

[0158] See also Figure 8 The present application as a whole may include a spatiotemporal fusion module based on epipolar constraints. After image acquisition, spatial convolution may be performed through a convolutional neural network to obtain spatial information of elements in the image (i.e., category information and position information of the elements). Then, an epipolar search may be performed to obtain the position range information of each element in another image. Then, based on the position range information of the image within a pixel pair and the position information of the target element in the image, spatiotemporal fusion is performed to output an image, which is an image after credibility detection of the category information of the element is completed. The image includes the location information and category information of the element.

[0159] After detecting the position information and category information of the elements, the present application can determine the pair of target elements with the same target category information, and then determine the position range information of the first target element in the second image where the second target element is located, and then based on the position information of the second target element and this position range information, perform credibility analysis on the target position information of the first target element and the second target element, so as to further determine whether the target category information is accurate based on the credibility analysis result, thereby improving the accuracy of image processing.

[0160] In order to better implement the image processing method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above image processing method. The meanings of the terms are the same as those in the above image processing method, and the specific implementation details can refer to the description in the method embodiment.

[0161] like Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of an image processing device provided by an embodiment of the present application, wherein the image processing device may include an element recognition module 301, a determination module 302, a calculation module 303 and a credibility analysis module 304, wherein:

[0162] The element recognition module 301 is used to obtain at least one frame of image, perform element recognition on at least one frame of image, and obtain location information and category information of at least one element in each frame of image;

[0163] A determination module 302, configured to determine at least one target element set according to category information of elements in all images, wherein the target element set includes at least one set of target element pairs having the same target category information, and the target element pairs include a first target element and a second target element;

[0164] A calculation module 303, used to calculate the position range information of the first target element in the second image, where the second image is the image where the second target element is located;

[0165] The credibility analysis module 304 is used to perform credibility analysis on the target category information of the target element pair based on the position range information and the position information of the second target element.

[0166] In some embodiments, the credibility analysis module includes a calculation submodule and a determination submodule, wherein:

[0167] A calculation submodule, used for calculating the initial credibility of the target category information of the target element pair based on the position range information and the position information of the second target element;

[0168] The determination submodule is used to determine the target credibility of the target category information of the second target element according to the initial credibility.

[0169] In some embodiments, the calculation submodule includes a calculation unit and a comparison unit, wherein:

[0170] A calculation unit, configured to calculate distance confidence information of the second target element based on the position range information and the position information of the second target element;

[0171] The comparison unit is used to compare the distance confidence information with a preset confidence threshold to determine the initial credibility of the target category information of the target element pair.

[0172] In some embodiments, the determination submodule is specifically used for:

[0173] Determine at least one candidate credibility of the target category information of the second target element, the candidate credibility includes credibility of other target element pairs to which the second target element belongs, the other target element pairs being target element pairs in the target element set;

[0174] Based on the initial credibility and the candidate credibility, the target credibility of the target category information of the second target element is determined.

[0175] In some embodiments, the calculation module includes a calculation submodule, a determination submodule and a fusion submodule, wherein:

[0176] A calculation submodule, used to calculate a first stereoscopic optical center when a first element image is taken, and to calculate a second stereoscopic optical center when a second element image is taken, wherein the first element image is an image where the first target element is located;

[0177] A determination submodule, used to determine the element stereo coordinates of the first target element in the real environment;

[0178] The fusion submodule is used to fuse the first stereo optical center, the second stereo optical center and the element stereo coordinates to obtain the position range information of the first target element in the second image.

[0179] In some embodiments, the determination submodule is specifically used for:

[0180] Acquire a first shooting position and a first shooting parameter when the first image is shot;

[0181] The first shooting position, the first shooting parameter and the position information of the first target element are fused to obtain the element stereo coordinates of the first target element in the real environment.

[0182] In some embodiments, the computing submodule is specifically used for:

[0183] Acquiring a first shooting position and a first shooting parameter when the first image is captured, and acquiring a second shooting position and a second shooting parameter when the second image is captured;

[0184] The first stereoscopic optical center when the first element image is photographed is calculated according to the first photographing position and the first photographing parameters, and the second stereoscopic optical center when the second image is photographed is calculated according to the second photographing position and the second photographing parameters.

[0185] In some embodiments, the element detection module is specifically used to:

[0186] Extracting features from at least one frame of image to obtain image features of each frame of image, where the image features include at least one feature point;

[0187] In each frame of the image, multiple candidate feature regions are determined with the feature point as the center;

[0188] Detect and classify the elements in the candidate element regions, determine the target element region, the location information of the target element region, and the category information of the elements in the target element region from the candidate element regions, so as to obtain the location information and category information of at least one element in each frame image.

[0189] In some embodiments, the element detection module includes a module specifically configured to:

[0190] The trained feature recognition model is used to perform feature recognition on at least one frame of image to obtain location information and category information of at least one feature in each frame of image.

[0191] In some embodiments, the image processing apparatus further comprises:

[0192] A sample acquisition module, used to acquire multiple sample images and sample labels corresponding to the sample images;

[0193] A construction module for constructing a feature recognition model;

[0194] The training module is used to train the feature recognition model based on the sample images and sample labels to obtain the trained feature recognition model.

[0195] In the present application, the element recognition module 30 acquires at least one frame of image, performs element recognition on at least one frame of image, and obtains the position information and category information of at least one element in each frame of image. Then, the determination module 302 determines at least one target element set based on the category information of the elements in all images. The target element set includes at least one group of target element pairs with the same target category information. The target element pair includes a first target element and a second target element. The calculation module 303 calculates the position range information of the first target element in the second image, and the second image is the image where the second target element is located. Finally, the credibility analysis module 304 performs credibility analysis on the target category information of the target element pair based on the position range information and the position information of the second target element.

[0196] After detecting the position information and category information of the elements, the present application can determine the pair of target elements with the same target category information, and then determine the position range information of the first target element in the second image where the second target element is located, and then based on the position information of the second target element and this position range information, perform credibility analysis on the target position information of the first target element and the second target element, so as to further determine whether the target category information is accurate based on the credibility analysis result, thereby improving the accuracy of image processing.

[0197] In addition, the embodiment of the present application also provides a computer device, which can be a terminal or a server. Fig.10 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0198] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.10The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0199] The processor 401 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it executes various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0200] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0201] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0202] The computer device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0203] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0204] Acquire at least one frame of image, perform element recognition on at least one frame of image, and obtain position information and category information of at least one element in each frame of image; determine at least one target element set based on the category information of elements in all images, the target element set includes at least one group of target element pairs with the same target category information, and the target element pair includes a first target element and a second target element; calculate the position range information of the first target element in the second image, the second image is the image where the second target element is located; based on the position range information and the position information of the second target element, perform credibility analysis on the target category information of the target element pair.

[0205] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0206] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations in the above-mentioned embodiments.

[0207] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0208] To this end, an embodiment of the present application further provides a storage medium in which a computer program is stored, and the computer program can be loaded by a processor to execute the steps in any image processing method provided in the embodiment of the present application. For example, the computer program can execute the following steps:

[0209] Acquire at least one frame of image, perform element recognition on at least one frame of image, and obtain position information and category information of at least one element in each frame of image; determine at least one target element set based on the category information of elements in all images, the target element set includes at least one group of target element pairs with the same target category information, and the target element pair includes a first target element and a second target element; calculate the position range information of the first target element in the second image, the second image is the image where the second target element is located; based on the position range information and the position information of the second target element, perform credibility analysis on the target category information of the target element pair.

[0210] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0211] Since the computer program stored in the storage medium can execute the steps in any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0212] The above is a detailed introduction to an image processing method, device, storage medium and computer equipment provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Acquire at least one frame of image, perform element recognition on the at least one frame of image, and obtain position information and category information of at least one element in each frame of image; Determine at least one target element set according to category information of elements in all images, wherein the target element set includes at least one group of target element pairs having the same target category information, wherein the target element pairs include a first target element and a second target element; Calculating position range information of the first target element in a second image, where the second image is an image where the second target element is located; Calculating distance confidence information of the second target element based on the position range information and the position information of the second target element; Comparing the distance confidence information with a preset confidence threshold to determine the initial credibility of the target category information of the target element pair; The target credibility of the target category information of the second target element is determined according to the initial credibility.

2. The method according to claim 1, characterized in that: The step of determining the target credibility of the target category information of the second target element according to the initial credibility includes: Determine at least one candidate credibility of the target category information of the second target element, the candidate credibility includes credibility of other target element pairs to which the second target element belongs, the other target element pairs being target element pairs in the target element set; Based on the initial credibility and the candidate credibility, the target credibility of the target category information of the second target element is determined.

3. The method according to claim 1, characterized in that: Calculating the position range information of the first target element in the second image includes: Calculating a first stereoscopic optical center when a first image is taken, and calculating a second stereoscopic optical center when a second image is taken, wherein the first image is an image where the first target element is located; Determining the three-dimensional coordinates of the first target element in the real environment; The first stereoscopic optical center, the second stereoscopic optical center and the element stereoscopic coordinates are fused to obtain position range information of the first target element in the second image.

4. The method according to claim 3, characterized in that The determining of the three-dimensional coordinates of the first target element in the real environment includes: Acquire a first shooting position and a first shooting parameter when the first image is shot; The first shooting position, the first shooting parameter and the position information of the first target element are fused to obtain the element stereo coordinates of the first target element in the real environment.

5. The method according to claim 3, characterized in that: The step of calculating the first stereoscopic optical center when the first image is captured, and calculating the second stereoscopic optical center when the second image is captured, comprises: Acquire a first shooting position and a first shooting parameter when the first image is shot, and acquire a second shooting position and a second shooting parameter when the second image is shot; The first stereoscopic optical center when the first image is captured is calculated according to the first shooting position and the first shooting parameters, and the second stereoscopic optical center when the second image is captured is calculated according to the second shooting position and the second shooting parameters.

6. The method according to claim 1, characterized in that The performing element recognition on the at least one frame of image to obtain location information and category information of at least one element in each frame of image includes: Performing feature extraction on the at least one frame of image to obtain image features of each frame of image, wherein the image features include at least one feature point; Determine multiple candidate element regions with the feature point as the center in each frame of the image; Element detection and classification are performed on the candidate element regions, and a target element region, position information of the target element region, and category information of elements in the target element region are determined from the candidate element regions, thereby obtaining position information and category information of at least one element in each frame image.

7. The method according to claim 1, characterized in that The performing element recognition on the at least one frame of image to obtain location information and category information of at least one element in each frame of image includes: The trained element recognition model is used to perform element recognition on the at least one frame of image to obtain location information and category information of at least one element in each frame of image.

8. The method according to claim 7, characterized in that The method further comprises: Acquire multiple sample images and sample labels corresponding to the sample images; Constructing feature recognition models; The element recognition model is trained based on the sample image and the sample label to obtain a trained element recognition model.

9. An image processing device, characterized in that: include: An element recognition module is used to obtain at least one frame of image, perform element recognition on the at least one frame of image, and obtain position information and category information of at least one element in each frame of image; A determination module, configured to determine at least one target element set according to category information of elements in all images, wherein the target element set includes at least one group of target element pairs having the same target category information, wherein the target element pairs include a first target element and a second target element; A calculation module, used for calculating the position range information of the first target element in a second image, where the second image is an image where the second target element is located; a credibility analysis module, configured to calculate distance confidence information of the second target element based on the position range information and the position information of the second target element; Comparing the distance confidence information with a preset confidence threshold to determine the initial credibility of the target category information of the target element pair; The target credibility of the target category information of the second target element is determined according to the initial credibility.

10. A storage medium, characterized in that: The storage medium stores a plurality of computer programs, and the computer programs are suitable for being loaded by a processor to execute the steps in any one of the methods of claims 1 to 8.

11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images