Semi-supervised vector map construction method and device, storage medium and equipment

Abstract

The invention provides a semi-supervised vector map construction method and device, a storage medium and equipment, and the method comprises the steps: inputting a current training image and historical map information into a teacher model, so as to generate a pseudo-label map; inputting the current training image into the student model, and performing loss function training by taking the pseudo-tag map as supervision; after the student model training is completed, inputting real-time acquired images into the trained student model to enable the student model to output a vector update map; and transmitting the vector updating map to a server side, so that the server side updates the historical map information based on the vector updating map. And semi-supervised training is performed on the student model through the pseudo-label map generated by the teacher model, so that the quality and reliability of map construction are improved, and the dependence on expensive labels is remarkably reduced. Therefore, the trained student model can complete vector map construction after semi-supervised training, the construction quality of the vector map is guaranteed, and the accuracy of historical map information updating is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, apparatus, storage medium and device for constructing a semi-supervised vector map. Background Art

[0002] The field of autonomous driving perception is currently developing towards higher precision and real-time performance. Key technologies include deep learning-based visual perception, as well as data fusion and processing for LiDAR, radar, and ultrasonic sensors. Deep neural networks have made significant progress in tasks such as object detection, tracking, and semantic segmentation.

[0003] With the rapid advancement of artificial intelligence (AI), the safety and accuracy requirements for autonomous driving are becoming increasingly stringent. Autonomous driving systems must possess high-performance environmental perception capabilities in complex driving environments. Therefore, the ability to understand the semantics of driving scenarios, and in particular, the ability to construct semantic vector maps of the driving environment, has become a key cornerstone for the practical application of autonomous driving technology. Summary of the Invention

[0004] The purpose of the present invention is to provide a semi-supervised vector map construction method, device, storage medium and equipment to improve the above problems.

[0005] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, an embodiment of the present invention provides a semi-supervised vector map construction method, the method comprising: Input the current training image and historical map information into the teacher model to generate a pseudo-label map; Inputting the current training image into the student model and performing loss function training using the pseudo label map as supervision; After the student model training is completed, the method further includes: Input the real-time collected image into the trained student model to make it output vectors to update the map; The vector updated map is transmitted to the server side so that the server side updates the historical map information based on the vector updated map.

[0006] In a second aspect, an embodiment of the present invention provides a semi-supervised vector map construction device, the device comprising: A first processing unit is configured to input a current training image and historical map information into a teacher model to generate a pseudo-label map; input the current training image into a student model, and perform loss function training using the pseudo-label map as supervision; The second processing unit is used to input the real-time collected image into the trained student model after the student model training is completed, so that it outputs a vector update map; and transmit the vector update map to the server side so that it updates the historical map information based on the vector update map.

[0007] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0008] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0009] Compared with the prior art, the embodiments of the present invention provide a semi-supervised vector map construction method, apparatus, storage medium and device, which input the current training image and historical map information into the teacher model to generate a pseudo-label map; input the current training image into the student model, and use the pseudo-label map as supervision to perform loss function training; after the student model training is completed, the real-time captured image is input into the trained student model to enable it to output a vector update map; the vector update map is transmitted to the server side so that it updates the historical map information based on the vector update map. The pseudo-label map generated by the teacher model is used to perform semi-supervised training on the student model, which improves the quality and reliability of map construction and significantly reduces the reliance on expensive annotations. The trained student model can complete the vector map construction after semi-supervised training, ensuring the quality of vector map construction and the accuracy of historical map information updates.

[0010] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0013] Figure 2 A schematic diagram of a flow chart of a semi-supervised vector map construction method provided in an embodiment of the present invention.

[0014] Figure 3 A schematic diagram of the units of a semi-supervised vector map construction device provided by an embodiment of the present invention.

[0015] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 501 - first processing unit; 502 - second processing unit. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0018] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0019] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0020] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limiting the present invention.

[0021] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed" and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections, or electrical connections; direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0022] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0023] This embodiment of the present invention provides a semi-supervised vector map construction method that combines the advantages of supervised and unsupervised learning, improving the accuracy and efficiency of map construction by leveraging limited labeled data and a large amount of unlabeled data. This method can incorporate temporal information for semi-supervised vector map construction, improving the quality and reliability of constructed maps while significantly reducing reliance on expensive annotations. Furthermore, the rational and efficient integration of temporal information enables robust map construction in both single-frame and temporal modes.

[0024] The embodiment of the present invention provides an electronic device, which can be a server device, a driving computer device, a mobile phone device or other intelligent terminal device. Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules stored in the memory 11, such as computer programs.

[0025] The processor 10 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the semi-supervised vector map construction method can be completed by hardware integrated logic circuits in the processor 10 or software instructions. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0026] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0027] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus 12 or one type of bus 12.

[0028] The memory 11 is used to store programs, such as a program corresponding to the semi-supervised vector map construction apparatus. The semi-supervised vector map construction apparatus includes at least one software functional module, which can be stored in the memory 11 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, the processor 10 executes the program to implement the semi-supervised vector map construction method.

[0029] Possibly, the electronic device provided by the embodiment of the present invention further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0030] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0031] The semi-supervised vector map construction method provided by the embodiment of the present invention can be applied to, but not limited to, Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 The semi-supervised vector map construction method includes a student model training stage and a running stage after the student model training is completed. Among them, the student model training stage includes S11 and S12, and the running stage after the student model training is completed includes S21 and S22, which are specifically explained as follows.

[0032] S11, input the current training image and historical map information into the teacher model to generate a pseudo-label map.

[0033] The historical map information may be stored locally or obtained from a server.

[0034] S12, input the current training image into the student model, and use the pseudo label map as supervision to perform loss function training.

[0035] S21, input the real-time collected image into the trained student model to make it output a vector to update the map.

[0036] The current training images and the implemented captured images are images captured by cameras installed on the vehicle (which may be, but are not limited to, the left front, left rear, right front, and right rear cameras) while the vehicle is driving.

[0037] S22: Transmit the vector updated map to the server, so that the server updates the historical map information based on the vector updated map.

[0038] It should be noted that the student model in the training phase can be a neural network or deep learning model that has undergone preliminary training on a small number of labeled sample images. Semi-supervised training of the student model using the pseudo-labeled map generated by the teacher model improves the quality and reliability of map construction and significantly reduces reliance on expensive annotations. This allows the trained student model to complete vector map construction after semi-supervised training, ensuring both the quality of vector map construction and the accuracy of historical map updates.

[0039] Based on the above, regarding the content of S11, the embodiment of the present invention also provides an optional implementation method, which is referred to below. S11, inputting the current training image and historical map information into the teacher model to generate a pseudo-label map, includes: S111, S112, S113 and S114, which are described in detail below.

[0040] S111, the teacher model performs (weak) enhancement processing on the current training image to obtain a first enhanced image.

[0041] Among them, (weak) enhancement processing refers to the restoration processing of the image in the direction of the weather environment (such as low-light enhancement and rain enhancement). It should be noted that the enhancement amplitude of weak enhancement processing is smaller than that of strong enhancement processing.

[0042] S112: The teacher model transforms the first enhanced image to obtain its corresponding first bird's-eye view feature.

[0043] Optionally, S112, the teacher model transforms the first enhanced image to obtain its corresponding first bird's-eye view feature, including: S112A, S112B and S112C, as follows.

[0044] S112A, the teacher model uses a first deep convolutional upscaling network (CNN) to extract features from the first enhanced image to obtain a first two-dimensional feature.

[0045] In an embodiment of the present invention, a deep convolutional upscaling network (CNN) can extract multi-level, deep two-dimensional feature information from an image.

[0046] S112B, the teacher model performs a three-dimensional transformation on the first two-dimensional feature to obtain a first three-dimensional feature.

[0047] S112C, the teacher model performs a bird's-eye view projection on the first three-dimensional feature to obtain a first bird's-eye view feature.

[0048] Through this process, the model can convert the planar information of the two-dimensional image into bird's-eye view (BEV) information, and more accurately model various objects in the environment and their positional relationships.

[0049] S113, the teacher model uses the query generator therein to process the historical map information to extract the target information.

[0050] The target information is the hierarchical content and location query results for each point in the target element. The target element is a map element (such as a lane centerline, lane markings, or pedestrian crossing) within a preset distance range from the current training image's capture location. The hierarchical content can be understood as semantic query information, including RGB values and the predicted category corresponding to the map element.

[0051] Optionally, in S113, the teacher model processes the historical map information using the query generator therein to extract target information, including: S113A and S113B, as follows.

[0052] S113A, the teacher model filters out the target area map from the historical map information based on the shooting location.

[0053] By screening, the target area map is cropped from the historical map information, thereby reducing the subsequent calculation amount.

[0054] S113B, the teacher model inputs the target area map (as reference information) into the query generator therein to obtain the target information.

[0055] Optionally, the target information is calculated as:

[0056]

[0057] in, Represents the hierarchical content of the bth point in the ath map element in the target area map. Indicates the location query result of the b-th point in the a-th map element in the target area map. represents the predicted category of the ath map element in the target area map, Represents the confidence of the ath map element in the target area map, Represents the current predicted position of the b-th point in the a-th map element in the target area map, Linear() represents the linear projection function, It represents the location query information of the b-th point in the a-th map element in the target area map, and NK() represents the position encoder.

[0058] Among them, the query generator is a module that generates query conditions based on previous prediction results. It helps generate queries related to maps and features, and then guides the model to learn how to better understand and locate targets in a specific environment.

[0059] S114: The teacher model inputs the target information and the first bird's-eye view feature into the first decoder inside it, so that it outputs pseudo-label maps with different confidence levels.

[0060] Optionally, in S114, the teacher model inputs the target information and the first bird's-eye view feature into the first decoder inside it, so that it outputs pseudo label maps with different confidence levels, including: S114A, S114B and S114C, as follows.

[0061] S114A: The first decoder determines a current decoding mode according to a random probability, where the current decoding mode is a single-frame mode or a continuous-time mode.

[0062] It should be understood that the first decoder randomly switches between the single frame mode and the continuous time mode, with the probability corresponding to the single frame mode being p and the probability corresponding to the continuous time mode being 1-p.

[0063] S114B: When the current decoding mode is the single-frame mode, the first decoder decodes the first bird's-eye view feature and outputs pseudo-label maps with different confidence levels.

[0064] S114C: When the current decoding mode is the continuous time mode, the first decoder decodes the target information and the first bird's-eye view feature, and outputs pseudo label maps with different confidence levels.

[0065] During the training phase, the query generator based on the previous prediction is selected according to whether the input is the first frame. If it is the first frame, it selects the single frame mode. If it is not the first frame, it randomly selects between the single frame mode and the time mode. This strategy can better combine the temporal information. By inputting these generated queries (layered content information and location query results ), the decoder that integrates historical queries can generate new map features, which are of great value for understanding the environment.

[0066] These pseudo-label maps are generated based on the model's ability to understand the environment and are evaluated through multiple levels of confidence to produce corresponding label outputs. The teacher model's main task is to provide accurate and reliable pseudo-label maps to the student model, thereby guiding the student model's training process.

[0067] Based on the above, regarding the content of S12, the embodiment of the present invention also provides an optional implementation method, please refer to the following. S12, input the current training image into the student model and use the pseudo label map as supervision to perform loss function training, including; S121, S122 and S123, as follows.

[0068] S121: The student model performs (strong) enhancement processing on the current training image to obtain a second enhanced image.

[0069] Among them, (strong) enhancement processing refers to repairing the image in the direction of the weather environment.

[0070] S122: The student model transforms the second enhanced image to obtain its corresponding second bird's-eye view feature.

[0071] Optionally, S122, the student model transforms the second enhanced image to obtain its corresponding second bird's-eye view feature, including: S122A, S122B and S122C, as follows.

[0072] S122A, the student model uses a second deep convolutional upscaling network (CNN) to extract features from the second enhanced image to obtain a second two-dimensional feature.

[0073] S122B, the student model performs a three-dimensional transformation on the second two-dimensional feature to obtain a second three-dimensional feature.

[0074] S122C, the student model performs a bird's-eye view projection on the second three-dimensional feature to obtain a second bird's-eye view feature.

[0075] S123, the student model determines a loss function based on the second bird's-eye view feature and the third bird's-eye view feature, and performs optimization training based on the loss function.

[0076] Among them, the third bird's-eye view feature is the bird's-eye view feature obtained by extracting features from the pseudo-label map with the highest confidence.

[0077] See also Figure 3 , Figure 3 An embodiment of the present invention provides a semi-supervised vector map construction device. Optionally, the semi-supervised vector map construction device is applied to the electronic device described above.

[0078] The semi-supervised vector map construction device includes: a first processing unit 501 and a second processing unit 502.

[0079] The first processing unit 501 is configured to input the current training image and historical map information into the teacher model to generate a pseudo-label map; input the current training image into the student model and perform loss function training using the pseudo-label map as supervision; The second processing unit 502 is used to input the real-time captured image into the trained student model after the student model training is completed, so that it outputs a vector update map; and transmit the vector update map to the server side so that it updates the historical map information based on the vector update map.

[0080] It should be noted that the semi-supervised vector map construction device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiment.

[0081] Embodiments of the present invention further provide a storage medium storing computer instructions or programs that, when read and executed, execute the semi-supervised vector map construction method of the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0082] The following provides an electronic device, which can be a server device, a driving computer device, a mobile phone device or other intelligent terminal device. Figure 1 As shown, the above-mentioned semi-supervised vector map construction method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the semi-supervised vector map construction method of the above-mentioned embodiment is performed.

[0083] In summary, the embodiments of the present invention provide a semi-supervised vector map construction method, apparatus, storage medium, and device, which input the current training image and historical map information into a teacher model to generate a pseudo-label map; input the current training image into a student model, and use the pseudo-label map as supervision to perform loss function training; after the student model training is completed, the real-time captured image is input into the trained student model to enable it to output a vector update map; the vector update map is transmitted to the server side so that it updates the historical map information based on the vector update map. The pseudo-label map generated by the teacher model is used to perform semi-supervised training on the student model, which improves the quality and reliability of map construction and significantly reduces the reliance on expensive annotations. The trained student model can complete the vector map construction after semi-supervised training, ensuring the quality of vector map construction and the accuracy of historical map information updates.

[0084] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0085] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A semi-supervised vector map construction method, characterized in that: The method comprises: Input the current training image and historical map information into the teacher model to generate a pseudo-label map; Inputting the current training image into the student model and performing loss function training using the pseudo label map as supervision; After the student model training is completed, the method further includes: Input the real-time collected image into the trained student model to make it output vectors to update the map; The vector updated map is transmitted to the server side so that the server side updates the historical map information based on the vector updated map.

2. The semi-supervised vector map construction method according to claim 1, wherein: The step of inputting the current training image and historical map information into the teacher model to generate a pseudo-label map includes: The teacher model performs enhancement processing on the current training image to obtain a first enhanced image; wherein the enhancement processing refers to repairing the image in the direction of the weather environment; The teacher model transforms the first enhanced image to obtain a first bird's-eye view feature corresponding thereto; The teacher model processes the historical map information using a query generator therein to extract target information, wherein the target information is a hierarchical content and location query result corresponding to each point in a target element, and the target element is a map element within a preset distance range of a shooting location of the current training image; The teacher model inputs the target information and the first bird's-eye view feature into a first decoder therein so that it outputs pseudo label maps with different confidence levels.

3. The semi-supervised vector map construction method according to claim 2, wherein: The teacher model transforms the first enhanced image to obtain a first bird's-eye view feature corresponding thereto, including: The teacher model uses a first deep convolutional upscaling network to perform feature extraction on the first enhanced image to obtain a first two-dimensional feature; The teacher model performs a three-dimensional transformation on the first two-dimensional feature to obtain a first three-dimensional feature; The teacher model performs a bird's-eye view projection on the first three-dimensional feature to obtain the first bird's-eye view feature.

4. The semi-supervised vector map construction method according to claim 2, wherein: The teacher model processes the historical map information using a query generator therein to extract target information, including: The teacher model filters out a target area map from the historical map information according to the shooting location; The teacher model inputs the target area map into a query generator therein to obtain the target information.

5. The semi-supervised vector map construction method according to claim 4, wherein: The formula for the target information is: in, Represents the hierarchical content of the bth point in the ath map element in the target area map, represents the position query result of the bth point in the ath map element in the target area map, represents the predicted category of the ath map element in the target area map, represents the confidence of the ath map element in the target area map, represents the current predicted position of the bth point in the ath map element in the target area map, Linear() represents the linear projection function, represents the location query information of the bth point in the ath map element in the target area map, and NK() represents a position encoder.

6. The semi-supervised vector map construction method according to claim 2, wherein: The teacher model inputs the target information and the first bird's-eye view feature into a first decoder therein, so that it outputs pseudo-label maps with different confidence levels, including: The first decoder determines a current decoding mode according to a random probability, wherein the current decoding mode is a single frame mode or a continuous time mode; When the current decoding mode is a single-frame mode, the first decoder decodes the first bird's-eye view feature and outputs pseudo label maps with different confidence levels; When the current decoding mode is the continuous time mode, the first decoder decodes the target information and the first bird's-eye view feature, and outputs pseudo label maps with different confidence levels.

7. The semi-supervised vector map construction method according to claim 1, wherein: The step of inputting the current training image into the student model and performing loss function training using the pseudo label map as supervision comprises: The student model performs enhancement processing on the current training image to obtain a second enhanced image; wherein the enhancement processing refers to repairing the image in the direction of the weather environment; The student model transforms the second enhanced image to obtain a second bird's-eye view feature corresponding thereto; The student model determines a loss function according to the second bird's-eye view feature and the third bird's-eye view feature, and performs optimization training according to the loss function; The third bird's-eye view feature is a bird's-eye view feature obtained by extracting features from the pseudo-label map with the highest confidence.

8. A semi-supervised vector map construction device, characterized in that: The device comprises: A first processing unit is configured to input a current training image and historical map information into a teacher model to generate a pseudo-label map; input the current training image into a student model, and perform loss function training using the pseudo-label map as supervision; The second processing unit is used to input the real-time collected image into the trained student model after the student model training is completed, so that it outputs a vector update map; and transmit the vector update map to the server side so that it updates the historical map information based on the vector update map.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: a processor and a memory, the memory being configured to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images