Model training method, scene category calibration method, equipment and readable medium
By processing feature labels on image samples and entering scene category calibration model for training, the problem of heavy calibration work in cross-data set object detection is solved, and the effect of reducing model training costs and improving cross-data set processing performance is achieved.
Patent Information
- Application Number
- CN202411467067.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-03-07
AI Technical Summary
The prior art requires heavy calibration work in cross-data set object detection, resulting in high cost of model training and the inability to effectively fuse multiple scene data sets for training.
By obtaining image samples in the sample dataset, performing feature label processing, converting image samples into numerical feature spatial data, generating training sample pairs, and inputting preset scene category calibration model for training until the model meets the preset conditions, to determine the scene category calibration model.
This reduces the cost of model training, improves the cross-dataset processing performance of the model, and realizes multi-scene and cross-dataset training based on a single model.
Smart Images
Figure CN119580250A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision, and in particular to a model training method, a scene category calibration method, a device and a readable medium. Background Art
[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
[0003] Cross-dataset target detection is of great significance in the application of engineering target detection. In actual business, target detection for a certain scene usually requires the following series of processes: first, the scene data needs to be calibrated, and then the detection model is trained and generated for a specific scene. As the business increases, the models also increase, which is very unfavorable for deployment costs; at the same time, conventional detection models cannot be trained by integrating multiple scene data sets.
[0004] In the related technology, for detection models that need to perform multiple detection tasks across data sets, during training, for uncalibrated data sets, calibration of multiple detection tasks is required; for different data sets that only calibrate a single detection task, cross-calibration of different detection tasks is required, and cross-calibration leads to heavy calibration work, and the heavy calibration work will increase the training cost of the detection model.
[0005] Therefore, there is an urgent need to propose a model training method, scene category calibration method, device and storage medium that can solve the above technical problems. Summary of the invention
[0006] Multiple aspects of the present application provide a model training method, a scene category calibration method, a device and a readable medium to reduce the model training cost and improve the model's cross-dataset processing performance.
[0007] In one aspect of the present application, a model training method is provided, wherein the model is applied to scene category calibration, and the method comprises:
[0008] Acquire an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples;
[0009] Performing feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs;
[0010] The training sample pairs are input into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
[0011] Furthermore, feature label processing is performed on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs, including: obtaining sample data subset information where any image sample is located, the sample data subset information including scene category and scene position information; determining the mask value of the image sample based on the scene category of the image sample; performing feature label processing on the scene category and the scene position information to determine the numerical feature space data of the image sample to obtain a source image sample; performing feature label processing on the mask value, the scene category and the scene position information to determine the numerical feature space data of the image sample to obtain a target image sample; and obtaining multiple training sample pairs based on the source image sample and the target image sample.
[0012] Furthermore, based on the scene category of the image sample, the mask value of the image sample is determined, including: obtaining the sample data subset and the total number N of the sample data subsets, and the scene category in any of the sample data subsets and the total number n of the scene categories; generating a first sequence based on the sample data subset, the first sequence containing N*n marking bits; determining the sample data subset where the scene category of the image sample is located, setting the corresponding marking position in the first sequence to 1, and generating the mask value of the image sample.
[0013] Furthermore, based on the scene category of the image sample, the mask value of the image sample is determined, including: obtaining the scene category and the total number m of scene categories contained in the sample data subset in the sample data set; generating a second sequence based on the scene category, the second sequence containing m marking positions; determining the scene category of the image sample, setting the corresponding marking position in the second sequence to 0, and generating the mask value of the image sample.
[0014] Furthermore, the training sample pair is input into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, including: inputting the source image sample in the training sample pair into a preset scene category calibration model for training to obtain a predicted scene category; obtaining a mask value of a target image sample in the training sample pair; enhancing the source image sample based on the predicted scene category and the mask value to obtain an enhanced source image sample; generating a training sample pair based on the enhanced source image sample and the target image sample; inputting the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions; wherein the preset conditions include: a loss value of the preset scene category calibration model until the loss value is not greater than a first threshold; and / or, the accuracy of the preset scene category calibration model is not lower than a third threshold; and / or, the recall rate of the preset scene category calibration model is not higher than a second threshold.
[0015] In another aspect of the present application, a scene category calibration method is provided, the method comprising: obtaining a target image whose scene category is to be calibrated; inputting the target image into a scene category calibration model to calibrate the scene category of the target image; wherein the scene category calibration model is trained by the model training method as described above.
[0016] In another aspect of the present application, a model training device is provided, the device is used to generate a scene category calibration model, the device comprises:
[0017] An acquisition unit, used for acquiring an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples;
[0018] A processing unit performs feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain a plurality of training sample pairs;
[0019] The training unit inputs the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
[0020] In another aspect of the present application, a scene category calibration device is provided, which uses the scene category calibration model generated by the model training device as described above to implement scene category calibration, and the device includes:
[0021] An acquisition unit, which acquires a target image to be calibrated for a scene category;
[0022] The calibration unit inputs the target image into a scene category calibration model to calibrate the scene category of the target image.
[0023] Another aspect of the present application provides an electronic device, the device comprising:
[0024] at least one processor; and
[0025] a memory communicatively connected to the at least one processor; wherein,
[0026] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method and / or the scene category calibration method as described above.
[0027] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the model training method and / or the scene category calibration method.
[0028] In another aspect of the present application, a computer program product is provided, including a computer program, which implements the model training method and / or the scene category calibration method when executed by a processor.
[0029] The model training method described in the solution provided in the embodiment of the present application obtains image samples in a sample data set, wherein the sample data set includes several sample data subsets, and the sample data subsets include several image samples; performs feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs; inputs the training sample pairs into a preset scene category calibration model for training until the preset scene category calibration model meets the preset conditions; so as to reduce the model training cost and improve the model cross-dataset processing performance;
[0030] Furthermore, in the technical solution for determining the mask value of the image sample based on the scene category of the image sample, a technical solution is creatively proposed to obtain the sample data subset and the total number N of the sample data subsets, and the scene category in any of the sample data subsets and the total number n of the scene categories; generate a first sequence based on the sample data subset, the first sequence contains N*n marker bits; determine the sample data subset where the scene category of the image sample is located, set the corresponding marker position in the first sequence to 1, and generate the mask value of the image sample. By adopting the static mask strategy design, the model can realize multi-scene and cross-dataset training, and the effect of cross-dataset training based on a single model is achieved; that is, a single model is used to replace multiple models, which greatly reduces the model training cost;
[0031] Furthermore, in the technical solution for determining the mask value of the image sample based on the scene category of the image sample, a technical solution for creatively proposing the following steps: obtaining the scene categories and the total number of scene categories m contained in the sample data subset in the sample data set; generating a second sequence based on the scene category, wherein the second sequence contains m marking positions; determining the scene category of the image sample, setting the corresponding marking position in the second sequence to 0, and generating the mask value of the image sample. Through the design of the dynamic mask strategy, the model realizes that the scene categories between different sample data subsets can supervise each other during the cross-dataset training process, thereby achieving the technical effect of improving the accuracy of the model prediction results. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0033] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0034] Figure 1 A flowchart of a model training method provided in one embodiment of the present application;
[0035] Figure 2 A schematic diagram of numerical feature space data provided by an embodiment of the present application;
[0036] Figure 3 A schematic diagram of determining a mask value based on a static mask strategy and a dynamic mask strategy provided in an embodiment of the present application;
[0037] Figure 4 A schematic diagram of the result of scene category calibration based on a trained scene category calibration model provided in an embodiment of the present application;
[0038] Figure 5 A schematic diagram of a flow chart of a scene category calibration method provided by another embodiment of the present application;
[0039] Figure 6 A schematic diagram of the structure of a model training device provided in yet another embodiment of the present application;
[0040] Figure 7 A schematic diagram of the structure of a scene category calibration device provided in yet another embodiment of the present application;
[0041] Figure 8A schematic diagram of the structure of an electronic device suitable for implementing the solution in the embodiments of the present application;
[0042] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0044] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.
[0045] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0046] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0047] The present application provides a model training method, which is applied to scene category calibration. The method includes:
[0048] Acquire an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples;
[0049] Performing feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs;
[0050] The training sample pairs are input into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
[0051] In actual scenarios, the execution subject of the method can be a user device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.
[0052] Embodiment 1
[0053] This application proposes a model training method using the YOLO framework. Figure 1 The processing flow of a model training method provided in an embodiment of the present application is shown, and the method at least includes the following processing steps:
[0054] Step S101, obtaining an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples;
[0055] Step S102, performing feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain a plurality of training sample pairs;
[0056] Step S103: input the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
[0057] In one of the embodiments, the method further includes: pre-generating a sample data set; specifically, pre-generating a sample data set includes: acquiring at least two different sample data subsets; any sample data subset includes at least two different scene categories.
[0058] In one embodiment, feature label processing is performed on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs, including: obtaining sample data subset information where any image sample is located, the sample data subset information including scene category and scene location information; determining a mask value of the image sample based on the scene category of the image sample; performing feature label processing on the scene category and the scene location information to determine the numerical feature space data of the image sample to obtain a source image sample; performing feature label processing on the mask value, the scene category and the scene location information to determine the numerical feature space data of the image sample to obtain a target image sample; and obtaining multiple training sample pairs based on the source image sample and the target image sample.
[0059] Specifically, the present application performs feature label processing on any image sample to convert the image sample into the corresponding branch flower feature space, and performs feature label processing on the scene category and scene position information, including: using the YOLO format to perform feature label processing on the scene category and scene position information, and, performing feature label processing on the scene category, scene position information and mask value. The generated numerical feature space data is as follows: Figure 2 shown. Figure 2 The last row in the figure represents the mask value added to the corresponding source pattern sample; except for the last row, the other rows represent the numerical feature space data corresponding to the source sample image. The first column in the numerical feature space data represents the scene category, and the next four columns represent the scene location information.
[0060] In one embodiment, in order to achieve cross-dataset training of a single model and reduce the model training cost, the present application creatively proposes a static mask strategy; further, based on the cross-dataset training of a single model, in order to further improve the overall prediction performance of the model, the present application creatively proposes a dynamic mask strategy. Figure 3 As shown in the figure, the mask value is determined based on the static mask strategy and the dynamic mask strategy.
[0061] Wherein, based on the static mask strategy, determining the mask value of the image sample includes: obtaining the sample data subset and the total number N of the sample data subsets, and the scene category in any of the sample data subsets and the total number n of the scene categories; generating a first sequence based on the sample data subset, the first sequence containing N*n marking bits; determining the sample data subset of the scene category of the image sample, setting the corresponding marking position in the first sequence to 1, and generating the mask value of the image sample.
[0062] Specifically, the sample data set includes two sample data subsets for illustration, the sample data set includes sample data subset A and sample data subset B, sample data subset A includes scene categories A1, A2 and A3, sample data subset B includes scene categories B1, B2 and B3, that is, the total number of sample data subsets is 2, and the total number of scene categories in any sample data subset is 3; a first sequence is generated based on the sample data subsets, and the first sequence is [A1, A2, A3, B1, B2, B3]; wherein A1, A2, A3 are scene categories corresponding to sample data subset A, and B1, B2, B3 are scene categories corresponding to sample data subset B; when the scene category of the image sample is A1, the mask value of the image sample is 1, 1, 1, 0, 0, 0; when the scene category of the pattern sample is B1, the mask value of the image sample is 0, 0, 0, 1, 1, 1. The initial value of each marker bit in the first sequence is 0.
[0063] In one embodiment, based on the dynamic mask strategy, the mask value of the image sample is determined, including: obtaining the scene category and the total number m of scene categories contained in the sample data subset in the sample data set; generating a second sequence based on the scene category, the second sequence containing m marking positions; determining the scene category of the image sample, setting the corresponding marking position in the second sequence to 0, and generating the mask value of the image sample.
[0064] Specifically, the sample data set includes two sample data subsets for illustration, the sample data set includes sample data subset A and sample data subset B, sample data subset A includes scene categories A1, A2 and A3, sample data subset B includes scene categories B1, B2 and B3, that is, the total number of sample data subsets is 2, and the total number of scene categories in any sample data subset is 3; a first sequence is generated based on the sample data subsets, and the first sequence is [A1, A2, A3, B1, B2, B]; wherein A1, A2, A3 are scene categories corresponding to sample data subset A, and B1, B2, B3 are scene categories corresponding to sample data subset B; when the scene category of the image sample is A1, the mask value of the image sample is 1, 0, 0, 0, 0, 0; when the scene category of the pattern sample is B1, the mask value of the image sample is 0, 0, 0, 1, 0, 0. Similarly, the initial value of each marker bit in the first sequence is 0.
[0065] In one of the embodiments, in order to ensure the prediction accuracy of the trained model and improve the prediction performance of the model, the training sample pair is input into a preset scene category calibration model for training until the preset scene category calibration model meets the preset conditions, including: inputting the source image samples in the training sample pair into a preset scene category calibration model for training to obtain the predicted scene category; obtaining the mask value of the target image sample in the training sample pair; enhancing the source image sample based on the predicted scene category and the mask value to obtain the enhanced source image sample; generating a training sample pair based on the enhanced source image sample and the target image sample; inputting the training sample pair into the preset scene category calibration model for training until the preset scene category calibration model meets the preset conditions.
[0066] Specifically, enhancing the source image sample based on the predicted scene category and the mask value to obtain the enhanced source image sample includes: multiplying the predicted scene category and the mask value to obtain a first result, performing TTA processing on the corresponding source image sample based on the first result, and obtaining the enhanced source image sample. The predicted scene category and the mask value have the same data type and the same data size.
[0067] In one embodiment, in order to ensure the accuracy of the scene category calibration model during the scene calibration process, the preset conditions include: the loss value of the preset scene category calibration model is not greater than a first threshold; and / or, the accuracy of the preset scene category calibration model is not lower than a third threshold; and / or, the recall rate of the preset scene category calibration model is not higher than a second threshold.
[0068] In one embodiment, obtaining the loss value of a preset scene category calibration model includes: determining a first loss value based on a source image sample and a target image sample in the training sample pair; and determining the loss value of the preset scene category calibration model based on the first loss value and the mask value.
[0069] Specifically, the loss value of the model is determined based on the first loss value and the mask value, that is: a mask value will be added to each image sample during the model training process, and while reading the scene category and scene position information of the pattern sample, the mask value of the image sample will also be read during the model training process; and then, when calculating the loss value of each training sample pair, the first loss value will be multiplied by the mask of the target image sample in the training sample pair to ensure the accuracy of the loss value and thus ensure the prediction performance of the trained model.
[0070] In one embodiment, in order to ensure that the trained model can accurately calibrate the scene category, this embodiment obtains a total of 14 scene categories (such as Figure 4 The model described in this application is tested on a sample data set of images (the information corresponding to the class in the image is the scene category). The test results are as follows: Figure 4 shown. Figure 4 In Box (P), it represents the accuracy of the detected bounding box, R represents the recall rate, and mAP50 is the accuracy index of the scene category calibration model (the higher the value corresponding to mAP50, the higher the accuracy of the model and the more accurate the prediction result). It has been verified through testing that the detection accuracy of the scene category calibration model proposed in this application in multi-scene category calibration meets the requirements of actual application scenarios; and it has been confirmed in actual tests that: compared with the single scene category calibration model, the scene category calibration result of the cross-domain scene category calibration model proposed in this application is extremely accurate.
[0071] Embodiment 2
[0072] like Figure 5 As shown, the present application proposes a scene category calibration method, the method comprising:
[0073] Step S201, obtaining a target image whose scene category is to be calibrated;
[0074] Step S202: input the target image into a scene category calibration model to calibrate the scene category of the target image; wherein the scene category calibration model is trained by the model training method described in the first embodiment.
[0075] Embodiment 3
[0076] like Figure 6 As shown, the present application proposes a model training device, which is used to generate a scene category calibration model, and the device includes:
[0077] An acquisition unit, used for acquiring an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples;
[0078] A processing unit performs feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain a plurality of training sample pairs;
[0079] The training unit inputs the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
[0080] Embodiment 4
[0081] like Figure 7 As shown, the present application proposes a scene category calibration device, which uses the scene category calibration model obtained by training in Example 3 to implement scene category calibration, and the device includes:
[0082] An acquisition unit, which acquires a target image to be calibrated for a scene category;
[0083] The calibration unit inputs the target image into a scene category calibration model to calibrate the scene category of the target image.
[0084] Embodiment 5
[0085] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the method corresponding to the electronic device may be the model training method and the scene category calibration method for scene category calibration in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.
[0086] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may be an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing.
[0087] Figure 8 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the device 800 includes a central processing unit (CPU, Central Processing Unit) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 802 or the program loaded from the storage part 808 to the random access memory (RAM, Random Access Memory) 803. In RAM803, various programs and data required for system operation are also stored. CPU 801, ROM 802 and RAM 803 are connected to each other through bus 804. Input / output (I / O, Input / Output) interface 805 is also connected to bus 804.
[0088] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 808 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet.
[0089] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 801, the above functions defined in the method of the present application are executed.
[0090] Embodiment 6
[0091] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application described above.
[0092] Specifically, the present embodiment may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device, or device.
[0093] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0094] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0095] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0097] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0099] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
[0101] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.
Claims
1. A model training method, wherein: The model is applied to scene category calibration, and the method comprises: Acquire an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples; Performing feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs; The training sample pairs are input into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
2. The model training method according to claim 1, wherein: Perform feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain multiple training sample pairs, including: Acquire sample data subset information where any image sample is located, wherein the sample data subset information includes scene category and scene location information; Determining a mask value of the image sample based on the scene category of the image sample; Performing feature label processing on the scene category and the scene position information, determining numerical feature space data of the image sample, and obtaining a source image sample; Performing feature label processing on the mask value, the scene category and the scene position information to determine the numerical feature space data of the image sample and obtain a target image sample; Based on the source image samples and the target image samples, a plurality of training sample pairs are obtained.
3. The model training method according to claim 2, wherein: Determining a mask value of the image sample based on the scene category of the image sample includes: Obtain the sample data subsets and the total number N of the sample data subsets, and the scene categories in any of the sample data subsets and the total number n of the scene categories; Generate a first sequence based on the sample data subset, wherein the first sequence includes N*n marker bits; Determine a sample data subset where the scene category of the image sample is located, set the corresponding mark position in the first sequence to 1, and generate a mask value of the image sample.
4. The model training method according to claim 2, wherein: Determining a mask value of the image sample based on the scene category of the image sample includes: Obtaining scene categories and a total number m of scene categories included in a sample data subset in the sample data set; Generate a second sequence based on the scene category, wherein the second sequence includes m marker bits; Determine the scene category of the image sample, set the corresponding mark position in the second sequence to 0, and generate a mask value of the image sample.
5. The model training method according to claim 3 or 4, wherein: Inputting the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, including: Inputting the source image samples in the training sample pairs into a preset scene category calibration model for training to obtain a predicted scene category; Obtaining a mask value of a target image sample in the training sample pair; enhancing the source image sample based on the predicted scene category and the mask value to obtain an enhanced source image sample; Generating a training sample pair based on the enhanced source image sample and the target image sample; Inputting the training sample pairs into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions; Among them, the preset conditions include: the loss value of the preset scene category calibration model is not greater than a first threshold; and / or the accuracy of the preset scene category calibration model is not lower than a third threshold; and / or the recall rate of the preset scene category calibration model is not higher than a second threshold.
6. A scene category calibration method, wherein: The method comprises: Obtain the target image to be calibrated for the scene category; Inputting the target image into a scene category calibration model to calibrate the scene category of the target image; Wherein, the scene category calibration model is obtained by training using the model training method described in any one of claims 1 to 5.
7. A model training device, wherein: The device is used to generate a scene category calibration model, and the device includes: An acquisition unit, used for acquiring an image sample in a sample data set, wherein the sample data set includes a plurality of sample data subsets, and the sample data subsets include a plurality of image samples; A processing unit performs feature label processing on any image sample to convert the image sample into corresponding numerical feature space data to obtain a plurality of training sample pairs; The training unit inputs the training sample pair into a preset scene category calibration model for training until the preset scene category calibration model meets preset conditions, thereby determining the scene category calibration model.
8. A scene category calibration device, wherein: The device uses the scene category calibration model obtained by training according to claim 7 to implement scene category calibration, and the device includes: An acquisition unit, which acquires a target image to be calibrated for a scene category; The calibration unit inputs the target image into a scene category calibration model to calibrate the scene category of the target image.
9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
10. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 6.