Training methods, devices, and storage media for object recognition models

Abstract

This application provides a training method, apparatus, and storage medium for an object recognition model, relating to the field of intelligent video surveillance technology, which helps to improve the adaptability of the trained object recognition model more efficiently. The method includes: acquiring first monitoring images of the same monitoring point at different times, a reference image including a target object, and annotation information; generating a fused image of the same monitoring point at different times based on the acquired first monitoring image and reference image, the fused image including the target object and the background of the first monitoring image; determining the annotation information as the annotation result of the fused image; iteratively training the current object recognition model based on the fused image and annotation result until the model converges, obtaining a first target object recognition model; the first target object recognition model is used to identify the target object in the first monitoring image of the same monitoring point.

Description

Technical Field

[0001] This application relates to the field of intelligent video surveillance technology, and in particular to training methods, devices and storage media for object recognition models. Background Technology

[0002] In the field of surveillance, object recognition is a crucial research area. Existing object recognition technologies typically use sample images to pre-train an object recognition model, which is then used to identify target objects in images with the target scene as the background. However, in practical applications, the sample images used for training may not be images with the target scene as the background. Therefore, the pre-trained object recognition model may produce inaccurate recognition results when identifying target objects in images with the target scene as the background.

[0003] Typically, to solve the above problems, an object recognition model suitable for the target scene can be retrained. However, extracting samples to retrain the object recognition model requires a lot of manual labor (e.g., manually cropping out the monitoring samples frame by frame). Summary of the Invention

[0004] This application provides a training method, apparatus, and storage medium for an object recognition model, which helps to improve the adaptability of the trained object recognition model more efficiently.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising: acquiring a first monitoring image of the same monitoring point at different times, a reference image including a target object, and annotation information (the annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than an accuracy threshold); then, generating a fused image of the same monitoring point at different times based on the acquired first monitoring image and reference image, the fused image including the target object and the background of the first monitoring image; determining the annotation information as the annotation result of the fused image; iteratively training the current object recognition model based on the fused image and the annotation result until the model converges to obtain a first target object recognition model, the first target object recognition model being used to recognize the target object in the first monitoring image of the same monitoring point.

[0007] To enable the current object recognition model to identify target objects in images acquired from the aforementioned monitoring points, this embodiment generates a fused image of the monitoring point at different times based on a first monitoring image and a reference image acquired from the monitoring point. The fused image includes the target object and the background of the first monitoring image. The first monitoring image can be an image acquired from the specific monitoring point that only includes the background, or an image acquired from the specific monitoring point that includes both the background and foreground. The annotation information of the reference image is used as the annotation result of the fused image, representing the recognition result of the target object in the reference image. Thus, by training the current object recognition model based on the fused image and its annotation result, the resulting first target object recognition model can be used to identify target objects in images acquired from the monitoring point without the need for manual acquisition of training sample images including target objects from the monitoring point. This saves labor costs and improves the adaptability of the trained object recognition model more efficiently.

[0008] In one possible implementation, the above-mentioned "generating fused images of the same monitoring point at different times based on the acquired first monitoring image and reference image" includes: acquiring the image of the target object in the reference image; and fusing the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain a fused image. In this way, according to the image fusion algorithm, the image of the target object can be fused into different positions in the background image of the first monitoring image, resulting in multiple different fused images of the same monitoring point at different times, further enriching the fused images used in training and thus improving the adaptability of the trained first target object recognition model.

[0009] In another possible implementation, the above-mentioned "fusing the first monitoring image and the target object image according to a preset image fusion algorithm to obtain a fused image" includes: fusing the first monitoring image and the target object image according to a preset image fusion algorithm to obtain an intermediate image; and performing data augmentation processing on the intermediate image to obtain the fused image. In this way, data augmentation is performed on the intermediate image obtained by fusing through the preset image fusion algorithm (e.g., adding noise to the fused image, adjusting the contrast of the fused image, adjusting the saturation of the fused image, cropping or scaling the fused image to obtain more fused images), further enriching the fused images used in training, thereby further improving the adaptability of the trained first target object recognition model.

[0010] In another possible implementation, the aforementioned "obtaining a reference image containing the target object and annotation information" includes: inputting a test image containing the target object into the current object recognition model to obtain a recognition result for each test image; using the target recognition result and the test images corresponding to recognition results with accuracy greater than an accuracy threshold as reference images; the target recognition result is the adjusted recognition result in response to the input adjustment operation; and using the target recognition result and the recognition results with accuracy greater than an accuracy threshold as annotation information. In this way, using the target recognition result and the recognition results with accuracy greater than an accuracy threshold as annotation information saves the cost of manual annotation and improves the adaptability of the trained object recognition model more efficiently.

[0011] In another possible implementation, the method further includes: acquiring a second monitoring image containing the object to be identified at the same monitoring point; inputting the second monitoring image into a first target object recognition model to obtain an intermediate recognition result of the object to be identified in the second monitoring image; in response to an adjustment operation on the intermediate recognition result, acquiring an adjusted intermediate recognition result, the adjusted intermediate recognition result being used to characterize whether the object to be identified is a target object; determining the adjusted intermediate recognition result as the annotation result of the second monitoring image; and iteratively training the first target object recognition model based on the second monitoring image and the annotation result of the second monitoring image to obtain a second target object recognition model. Thus, in the process of applying the first target object recognition model to the recognition of the second detection image at a specific monitoring point, adjusting the obtained recognition result to obtain the adjusted intermediate recognition result is equivalent to defining the object to be identified in the second detection image, thereby making the trained second target object recognition model more accurate in identifying the object to be identified.

[0012] Secondly, embodiments of this application provide a training apparatus for an object recognition model. The training apparatus includes an acquisition module, a generation module, a determination module, and a training module. The acquisition module is used to acquire a first monitoring image of the same monitoring point at different times, a reference image including a target object, and annotation information; the annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than an accuracy threshold. The generation module is used to generate a fused image of the same monitoring point at different times based on the first monitoring image and the reference image acquired by the acquisition module; the fused image includes the target object and the background of the first monitoring image. The determination module is used to determine the annotation information acquired by the acquisition module as the annotation result of the fused image generated by the generation module. The training module is used to iteratively train the current object recognition model based on the fused image generated by the generation module and the annotation result determined by the determination module until the model converges, obtaining a first target object recognition model; the first target object recognition model is used to recognize the target object in the first monitoring image of the same monitoring point.

[0013] Optionally, the acquisition module is further configured to: acquire an image of the target object in the reference image; and the generation module is specifically configured to fuse the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain a fused image.

[0014] Optionally, the above-mentioned generation module is specifically used to: fuse the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain an intermediate image; and perform data enhancement processing on the intermediate image to obtain a fused image.

[0015] Optionally, the acquisition module is specifically used for: inputting a test image containing the target object into the current object recognition model to obtain the recognition result of each test image; using the target recognition result and the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold as reference images; the target recognition result is the adjusted recognition result in response to the input adjustment operation; and using the target recognition result and the recognition result with an accuracy greater than the accuracy threshold as annotation information.

[0016] Optionally, the acquisition module is further configured to: acquire second monitoring images of the same monitoring point at different times that include the object to be identified; input the second monitoring images into the first target object recognition model to obtain intermediate recognition results of the object to be identified in the second monitoring images; in response to the adjustment operation of the intermediate recognition results, acquire the adjusted intermediate recognition results, wherein the adjusted intermediate recognition results are used to characterize whether the object to be identified is the target object; determine the adjusted intermediate recognition results as the annotation results of the second monitoring images; and iteratively train the first target object recognition model based on the second monitoring images and the annotation results of the second monitoring images to obtain the second target object recognition model.

[0017] Thirdly, this application provides a training apparatus for an object recognition model, comprising: a memory and a processor; the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the training apparatus for the object recognition model executes the object recognition model training method provided in the first aspect and any possible implementation thereof.

[0018] Fourthly, this application provides a computer-readable storage medium including instructions. When the instructions are executed on a training device for an object recognition model, the training device for the object recognition model performs the object recognition model training method provided in the first aspect and any possible implementation thereof.

[0019] Fifthly, this application provides a computer program product that, when run on a training device for an object recognition model, causes the training device for the object recognition model to perform the training method as described in the design of the first aspect.

[0020] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the training device for the object recognition module, or it may be packaged separately from the processor of the training device for the object recognition model; this application does not impose any limitations on this.

[0021] The descriptions of the second, third, fourth, and fifth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0022] In this application, the name of the training device for the aforementioned object recognition model does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0023] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating the background and target object provided for embodiments of this application;

[0026] Figure 2 This application provides a schematic diagram of the structure of a training system according to an embodiment of the present application.

[0027] Figure 3 A flowchart illustrating a training method for an object recognition model provided in an embodiment of this application;

[0028] Figure 4 A schematic diagram of a test image provided in an embodiment of this application;

[0029] Figure 5This is a schematic diagram illustrating an embodiment of acquiring an image of a target object.

[0030] Figure 6 This is a schematic diagram illustrating how a first monitoring image and an image of a target object are fused according to a preset image fusion algorithm to obtain a fused image, as provided in an embodiment of this application.

[0031] Figure 7 A flowchart illustrating a method for optimizing a first target object recognition model to obtain a second target object recognition model, provided in an embodiment of this application;

[0032] Figure 8 A schematic diagram of a second monitoring image provided in an embodiment of this application;

[0033] Figure 9 A schematic diagram of the structure of a training device 30 for an object recognition model provided in an embodiment of this application;

[0034] Figure 10 A schematic diagram of the structure of a training device 30 for another object recognition model provided in an embodiment of this application;

[0035] Figure 11 A schematic diagram of the structure of a computer program product for training an object recognition model provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity or execution order.

[0038] To facilitate understanding of this application, the relevant terms involved in the embodiments of this application will be explained below.

[0039] Background: Refers to regions in an image that remain unchanged over a long timescale, acquired by an image acquisition device. For example... Figure 1 As shown in 'a', the background can be a room with no foreground.

[0040] Target object: refers to the area in an image acquired by an image acquisition device that changes over a longer time scale; it can also be called the foreground object. For example... Figure 1 As shown in b, the target object can be a human body that has fallen to the ground.

[0041] The structure of the training system to which the training method for the object recognition model provided in the embodiments of this application applies is described below.

[0042] Figure 2 This is a schematic diagram of the structure of a training system provided in an embodiment of this application. Figure 2 As shown, the training system may include: at least one image acquisition device 10 ( Figure 2 (The image acquisition devices are illustrated using cameras 10-1 to 10-3 as examples) and server 20.

[0043] The image acquisition device 10 can be used to acquire images of a designated area (such as a monitored room) and send the acquired images to the server 20. For example, when the designated area is a specific area inside a room in a senior living apartment, and the image acquisition device 10 is a camera, the image acquired by the image acquisition device 10 of the designated area is as follows: Figure 1 As shown.

[0044] Specifically, in practical applications, when the installation location (also known as the monitoring point) of the camera is fixed, its acquisition area is also fixed. For example, whether the camera is a bullet camera, a dome camera, or a large dome camera, the angle at which it acquires the image is also unique.

[0045] Server 20 stores a trained object recognition model, which is trained based on multiple labeled sample images. Server 20 can input a reference image including the target object into the object recognition model to obtain a prediction result, and then obtain the recognition result of the reference image including the target object based on the prediction result. Server 20 can be used to receive images of a specified area acquired by at least one image acquisition device 10, and generate a fused image based on the image of the specified area and the reference image including the target object. Server 20 can train the current object recognition model according to the training method of the object recognition model provided in the embodiments of this application based on the recognition result of the fused image and the reference image, thereby obtaining an image capable of recognizing the target object in an image with the image of the specified area as the background.

[0046] Specifically, the server 20 provided in this application embodiment can be a computer device such as a personal computer, laptop, smartphone, tablet, server, or server cluster. The image acquisition device 10 can be a device for acquiring images, such as a camera, snapshot camera, or video camera.

[0047] The training device for the object recognition model in the embodiments of this application may be Figure 2 The server 20 shown may also be a part of the device within server 20. For example, a chip system within server 20. This chip system supports server 20 in implementing the functions involved in the first aspect and any of its possible implementations. For example, acquiring an image of a designated area captured by at least one image acquisition device 10. The chip system includes a chip, but may also include other discrete devices or circuit structures.

[0048] The following combination Figure 2 The system architecture shown, taking the object recognition model training device as server 20 as an example, introduces the training method of the object recognition model provided in the embodiments of this application.

[0049] Before training an object recognition model suitable for the target scene, server 20 can acquire multiple labeled sample images and train the current object recognition model based on the acquired labeled sample images. The current object recognition model can be used to identify target objects in images. For example, server 20 trains the current object recognition model based on multiple images of target objects under a general background. This application embodiment does not limit the general background; exemplary general backgrounds can be green screens, street scenes, or indoor scenes, etc.

[0050] like Figure 3 The image shows a training method for an object recognition model provided in an embodiment of this application. The training method includes:

[0051] S11, Server 20 acquires the first monitoring image of the same monitoring point at different times, including a reference image of the target object and annotation information.

[0052] The annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than the accuracy threshold.

[0053] Server 20 acquires the first monitoring images of the same monitoring point at different times in the following ways:

[0054] In one possible implementation, server 20 receives a first monitoring image sent by image acquisition device 10. Specifically, after the monitoring points of image acquisition device 10 are determined, the acquisition area of ​​image acquisition device 10 is a fixed area of ​​the first scene. Image acquisition device 10 determines the first monitoring images from the acquired images of the first scene, using the first scene as the background area, and images without any foreground objects acquired at different time periods. The first monitoring image refers to the image of the area in the first scene acquired by image acquisition device 10 within a preset time period that does not change.

[0055] For example, after the image acquisition device 10 is set up at the monitoring point of the site to be monitored, the image acquisition device 10 first filters the images of the first scene acquired within a preset time period (e.g., within one day) to obtain the first monitoring images of the first scene under different times or different lighting conditions.

[0056] In another possible implementation, the server 20 determines the first monitoring image of the first scene from the image of the first scene sent by the image acquisition device 10 at the monitoring point to be monitored.

[0057] For example, after the image acquisition device 10 is set up at the monitoring point of the site to be monitored, the image acquisition device 10 sends images of the first scene acquired within a preset time period to the server 20. The server 20 then filters and processes the images of the first scene to obtain the first monitoring image of the first scene. Alternatively, the server 20 uses a background image extraction algorithm to obtain the background image in the first scene image and uses the obtained background image as the first monitoring image.

[0058] Server 20 obtains a reference image containing the target object and annotation information through the following steps:

[0059] Step 1: Server 20 obtains a test image containing the target object.

[0060] In one possible implementation, server 20 receives a test image containing the target object sent by another device (e.g., image acquisition device 10).

[0061] In another possible implementation, server 20 reads a local test image containing the target object.

[0062] Step 2: Server 20 inputs the test image containing the target object into the current object recognition model to obtain the recognition result for each test image.

[0063] Step 3: Server 20 uses the target recognition result and the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold as reference images. The target recognition result is the adjusted recognition result in response to the input adjustment operation.

[0064] Specifically, server 20 uses the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold in the recognition results of the test image as the reference image. For the recognition result with an accuracy less than or equal to the accuracy threshold in the recognition results, it obtains the adjusted recognition result as the target recognition result in response to the adjustment operation of the recognition result, and also uses the test image corresponding to the target recognition result as the reference image.

[0065] In one example, such as Figure 4The image shown is a test image of a person performing yoga. Server 20 inputs this test image into its current object recognition model, resulting in a recognition result of a person falling to the ground. The accuracy of this recognition result is 75%, which is lower than the accuracy threshold of 85%. During manual review of the recognition result, server 20 responds to input adjustments to the result, obtaining an adjusted recognition result of a person not falling to the ground. The target recognition result obtained by server 20 is a person not falling to the ground. Figure 4 The test image shown is the test image corresponding to the target recognition result. Server 20 will... Figure 4 The test image shown is used as a reference image.

[0066] Step 4: The server uses the target recognition results and recognition results with an accuracy greater than the accuracy threshold as annotation information for the reference image corresponding to each recognition result.

[0067] Based on the example in step three, server 20 considers the target object not to be a human body that has fallen to the ground. Figure 4 The annotation information for the reference image shown.

[0068] S12, Server 20 generates a fused image of the same monitoring point at different times based on the acquired first monitoring image and reference image. The fused image includes the target object and the background of the first monitoring image.

[0069] Specifically, server 20 acquires the image of the target object in the reference image and fuses the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain a fused image.

[0070] In one example, server 20 pairs, such as Figure 5 The image of the target object in a is used to obtain the image of the target object in b using a matting algorithm, as shown. Figure 6 The diagram shows how a first monitoring image a and an image b of the target object are fused according to a preset image fusion algorithm to obtain a fused image c.

[0071] Optionally, server 20 performs data augmentation on the obtained fused images to obtain more fused images.

[0072] For example, server 20 performs data augmentation processing on the obtained fused image, such as adding noise, adjusting the contrast of the fused image, adjusting the saturation of the fused image, cropping the fused image, or scaling the fused image, in order to obtain more fused images.

[0073] S13: Server 20 determines the annotation information as the annotation result of the fused image.

[0074] based on Figure 6 The fused image shown is labeled as a human body that has fallen to the ground.

[0075] Understandably, server 20 can automatically generate multiple fused images and their annotation results by executing S11 to S13, which theoretically reduces the manual cost of obtaining sample images and their annotation information for training the current object recognition model.

[0076] S14. Server 20 iteratively trains the current object recognition model based on the fused image and annotation results until the model converges, obtaining the first target object recognition model. The first target object recognition model is used to identify target objects in the first monitoring image of the same monitoring point.

[0077] Understandably, the server can also use the first target object recognition model as the new current object recognition model, and re-execute S11-S14 to iteratively train the current object recognition model to obtain a new first target object recognition model. Theoretically, the new first target object recognition model obtained will be more accurate in recognizing target objects in images containing target objects acquired at the detection point.

[0078] To enable the current object recognition model to identify target objects in images acquired from specific monitoring points, this embodiment generates a fused image of the monitoring point at different times based on a first monitoring image and a reference image acquired from that monitoring point. The fused image includes the target object and the background of the first monitoring image. The first monitoring image can be an image acquired from the specific monitoring point that only includes the background, or an image acquired from the specific monitoring point that includes both the background and the foreground. The annotation information of the reference image is determined as the annotation result of the fused image. The annotation information of the reference image represents the recognition result of the target object in the reference image. Thus, by training the current object recognition model based on the fused image and its annotation result, the resulting first target object recognition model can be used to identify target objects in images acquired from that monitoring point without the need for manual acquisition of training sample images including target objects from that monitoring point. This saves labor costs and improves the adaptability of the trained object recognition model more efficiently.

[0079] Subsequently, the server 20 can also acquire second monitoring images of the monitoring point at different times, including the object to be identified, and input the second monitoring images into the first target object recognition model to obtain intermediate recognition results of the object to be identified in the second monitoring images. In response to the adjustment operation of the intermediate recognition results, the server can acquire the adjusted intermediate recognition results, and optimize the first target object recognition model based on the adjusted intermediate recognition results and the second monitoring images to obtain the second target object recognition model.

[0080] like Figure 7 As shown, Figure 7 A flowchart illustrating a method for optimizing a first target object recognition model to obtain a second target object recognition model is provided. The method includes:

[0081] S21, Server 20 acquires a second monitoring image of the same monitoring point, including the object to be identified.

[0082] Specifically, server 20 receives a second monitoring image sent by image acquisition device 10, which has the first scene as the background area and includes the object to be identified.

[0083] For example, the second monitoring image acquired by server 20 includes, as shown below: Figure 8 As shown in Figures a and b. Figure 8 In section a, 80 represents the image of the object to be identified. Figure 8 In section b, 81 represents the image of the object to be identified.

[0084] It should be noted that the object to be identified and the target object mentioned above can be the same object or different objects, and this application embodiment does not limit this.

[0085] S22, Server 20 inputs the second monitoring image into the first target object recognition model to obtain the intermediate recognition result of the object to be recognized in the second monitoring image.

[0086] Based on the example in S21, Figure 8 In case a, the intermediate recognition result of the object to be identified is a human body that is not lying on the ground. Figure 8 The intermediate recognition result of the object to be identified in b is a human body that has fallen to the ground.

[0087] S23. In response to the adjustment operation on the intermediate identification result, server 20 obtains the adjusted intermediate identification result. The adjusted intermediate identification result is used to characterize whether the object to be identified is the target object.

[0088] Based on the example in S22, Figure 8 In the intermediate recognition result of the object to be identified in section a, the adjusted result is a human body that has fallen to the ground. Figure 8 The adjusted intermediate recognition result of the object to be identified in b is a human body that is not lying on the ground.

[0089] It is understood that the above-mentioned second monitoring image can be multiple second monitoring images, each of which includes the object to be identified. For each second monitoring image, the above-mentioned S22 to S23 can be executed to obtain the intermediate identification result corresponding to each second monitoring image. If the intermediate identification result is incorrect, the adjusted intermediate identification result can be obtained.

[0090] For example, suppose that the first target object recognition model can be used to identify object A, object B, and object C. The intermediate recognition results of the objects to be identified in the second monitoring images obtained by inputting multiple second monitoring images into the first target object recognition model, and the adjusted intermediate recognition results are shown in Table 1 below:

[0091] Table 1

[0092] Intermediate recognition results Adjusted intermediate recognition results A Aˊ Aˊ A Bˊ B B Bˊ C Cˊ Cˊ C

[0093] In Table 1, A represents object A, and Aˊ represents non-object A. B represents object B, Bˊ represents non-object B, C represents object C, and Cˊ represents non-object C.

[0094] S24, Server 20 determines the adjusted intermediate recognition result as the annotation result of the second monitoring image.

[0095] It is understandable that server 20 can obtain multiple second monitoring images and the annotation results of each second monitoring image by executing the above S21 to S24.

[0096] S25, Server 20 iteratively trains the first target object recognition model based on the second monitoring image and the annotation results of the second monitoring image to obtain the second target object recognition model.

[0097] It should be noted that, after obtaining multiple second monitoring images and the annotation results of each second monitoring image, server 20 can iteratively train the first target object recognition model based on the multiple second monitoring images and the annotation results of each second monitoring image to obtain a second target object recognition model; alternatively, server 20 can optimize the first target object recognition model based on each second monitoring image and its annotation results after obtaining each second monitoring image and its annotation results to obtain a second target object recognition model. This embodiment of the application does not limit this approach. It is understood that iteratively training the first target object recognition model based on the multiple second monitoring images and the annotation results of each second monitoring image after obtaining multiple second monitoring images results in a more stable second target object recognition model.

[0098] In this embodiment of the application, during the process of applying the first target object recognition model to the recognition of target objects in the first scene, the incorrect recognition results are manually adjusted (i.e., the adjusted intermediate recognition results are obtained in response to the adjustment operation of the intermediate recognition results). The adjusted recognition results are then used as the annotation results of the corresponding second monitoring image to iteratively train the first target object recognition model to obtain the second target object recognition model. Since the second target object recognition model has learned the adjusted intermediate recognition results, the second target object recognition model theoretically has a higher accuracy in recognizing target objects in the first scene than the first target object recognition model.

[0099] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] This application embodiment can divide the training device for the object recognition model into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0101] like Figure 9 The diagram shown is a structural schematic of a training device 30 for an object recognition model provided in an embodiment of this application. The training device 30 for the object recognition model includes an acquisition module 301, a generation module 302, a determination module 303, and a training module 304. The acquisition module 301 is used to acquire a first monitoring image of the same monitoring point at different times, a reference image including the target object, and annotation information; the annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than an accuracy threshold. For example: combined with... Figure 3 The acquisition module 301 can be used to execute S11, in conjunction with Figure 7The acquisition module 301 can be used to execute S21 to S23. The generation module 302 is used to generate a fused image of the same monitoring point at different times based on the first monitoring image and the reference image acquired by the acquisition module 301. The fused image includes the target object and the background of the first monitoring image. For example: combining Figure 3 The generation module 301 can be used to execute S12. The determination module 303 is used to determine the annotation information obtained by the acquisition module 301 as the annotation result of the fused image generated by the generation module 302, for example: combining... Figure 3 The determination module 301 can be used to execute S13. The training module 304 is used to iteratively train the current object recognition model based on the fused image and annotation results generated by the generation module 302 until the model converges, obtaining a first target object recognition model; the first target object recognition model is used to identify target objects in the first monitoring image of the same monitoring point. For example: combined with Figure 3 Training module 304 can be used to execute S14, combined with Figure 7 Training module 304 can be used to execute S24.

[0102] Optionally, the acquisition module 301 is further configured to: acquire an image of the target object in the reference image. The generation module 302 is specifically configured to fuse the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain a fused image.

[0103] Optionally, the generation module 302 is specifically used to: fuse the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain an intermediate image; and perform data enhancement processing on the intermediate image to obtain a fused image.

[0104] Optionally, the acquisition module 301 is specifically used for: inputting a test image containing the target object into the current object recognition model to obtain the recognition result of each test image; using the target recognition result and the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold as reference images; the target recognition result is the adjusted recognition result in response to the input adjustment operation; and using the target recognition result and the recognition result with an accuracy greater than the accuracy threshold as annotation information.

[0105] Optionally, the acquisition module 301 is further configured to: acquire a second monitoring image containing the object to be identified at the same monitoring point; input the second monitoring image into the first target object recognition model to obtain intermediate recognition results of the object to be identified in the second monitoring image; in response to the adjustment operation of the intermediate recognition results, acquire the adjusted intermediate recognition results, the adjusted intermediate recognition results being used to characterize whether the object to be identified is the target object; and determine the adjusted intermediate recognition results as the annotation results of the second monitoring image. The training module 304 is further configured to: iteratively train the first target object recognition model based on the second monitoring image and the annotation results of the second monitoring image to obtain a second target object recognition model.

[0106] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and their functions will not be repeated here.

[0107] Figure 10 This is a schematic diagram of the structure of a training device 30 for an object recognition model provided in an embodiment of this application, as shown below. Figure 10 As shown, the training device 30 for the object recognition model may include at least one processor 51, a memory 52, a communication interface 53, and a communication bus 54.

[0108] The following is combined Figure 10 The various components of the training device for the object recognition module are described in detail below:

[0109] The processor 51 is the control center of the training device for the object recognition module. It can be a single processor or a collective term for multiple processing elements. For example, the processor 51 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more DSPs, or one or more field-programmable gate arrays (FPGAs).

[0110] In a specific implementation, as one example, processor 51 may include one or more CPUs, for example... Figure 10 CPU0 and CPU1 are shown in the diagram. Furthermore, as one embodiment, the training device for the object recognition module may include multiple processors, such as... Figure 10The processors 51 and 55 are shown. Each of these processors can be a single-core processor or a multi-core processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0111] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0112] In a specific implementation, memory 52 is used to store data from this application and execute the software program of this application. Processor 51 can perform various functions of the object recognition model training device by running or executing the software program stored in memory 52 and calling the data stored in memory 52.

[0113] Communication interface 53 uses any transceiver-like device for communicating with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, and the cloud. Communication interface 53 may include a receiving unit to implement receiving functions and a transmitting unit to implement transmitting functions.

[0114] The communication bus 54 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0115] As an example, combined Figure 9 The receiving function of the acquisition module 301 in the training device 30 of the object recognition module can be provided by... Figure 10 The communication interface 53 in the processor 51 can be used to implement the processing functions of the acquisition module 301, the generation module 302, the determination module 303, and the training module 304. All of these functions can be implemented by the processor 51 calling the computer program in the storage unit 103.

[0116] Another embodiment of this application provides a computer-readable storage medium storing instructions that, when executed on a training device for an object recognition model, cause the training device to perform the method shown in the above-described method embodiment.

[0117] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0118] Figure 11 A conceptual partial view of a computer program product provided in an embodiment of this application is schematically shown. The computer program product includes a computer program for executing computer processes on a training device for an object recognition model.

[0119] In one embodiment, a computer program product is provided using a signal bearer medium 410. The signal bearer medium 410 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 3 , Figure 7 The described function or part of the function. Therefore, for example, refer to... Figure 3 In the embodiment shown, one or more features of S11-S14 can be provided by one or more instructions associated with the signal carrying medium 410. (See reference...) Figure 7In the embodiment shown, one or more features of S21-S24 can be fulfilled by one or more instructions associated with the signal carrying medium 410. Furthermore, Figure 11 The program instructions in the document also describe example instructions.

[0120] In some examples, the signal carrying medium 410 may include a computer-readable medium 411, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0121] In some implementations, the signal carrying medium 410 may include a computer recordable medium 412, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0122] In some implementations, the signal carrying medium 410 may include a communication medium 413, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0123] The signal-bearing medium 410 can be transmitted by a wireless communication medium 413 (e.g., a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocols). One or more program instructions can be, for example, computer-executable instructions or logical implementation instructions.

[0124] In some examples, such as targeting Figure 3 The described server 20 can be configured to provide various operations, functions, or actions in response to one or more program instructions transmitted via a computer-readable medium 411, a computer-recordable medium 412, and / or a communication medium 413.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0127] The modules described as separate components may or may not be physically separate. Components shown as units may be one or more physical units, meaning they may be located in one place or distributed across multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0128] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional module and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an object recognition model, characterized in that, The training method includes: The method involves acquiring first monitoring images of the same monitoring point at different times, a reference image including a target object, and annotation information. The annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than an accuracy threshold. Acquiring the reference image including the target object and the annotation information includes: inputting a test image including the target object into the current object recognition model to obtain the recognition result of each test image; using the target recognition result and the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold as the reference image; the target recognition result is the adjusted recognition result in response to an input adjustment operation; and using the target recognition result and the recognition result with an accuracy greater than the accuracy threshold as the annotation information. Based on the acquired first monitoring image and reference image, a fused image of the same monitoring point at different times is generated, the fused image including the target object and the background of the first monitoring image; The annotation information is determined as the annotation result of the fused image; The current object recognition model is iteratively trained based on the fused image and the annotation results until the model converges, resulting in a first target object recognition model; the first target object recognition model is used to identify the target object in the first monitoring image of the same monitoring point. The method further includes: Acquire a second monitoring image of the same monitoring point including the object to be identified; The second monitoring image is input into the first target object recognition model to obtain the intermediate recognition result of the object to be identified in the second monitoring image; In response to the adjustment operation on the intermediate identification result, an adjusted intermediate identification result is obtained; the adjusted intermediate identification result is used to characterize whether the object to be identified is the target object; The adjusted intermediate recognition result is determined as the annotation result of the second monitoring image; The first target object recognition model is iteratively trained based on the second monitoring image and the annotation results of the second monitoring image to obtain the second target object recognition model.

2. The training method according to claim 1, characterized in that, The step of generating a fused image of the same monitoring point at different times based on the acquired first monitoring image and reference image includes: Obtain an image of the target object from the reference image; According to a preset image fusion algorithm, the first monitoring image and the image of the target object are fused to obtain the fused image.

3. The training method according to claim 2, characterized in that, The step of fusing the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain the fused image includes: According to a preset image fusion algorithm, the first monitoring image and the image of the target object are fused to obtain an intermediate image; The intermediate image is subjected to data augmentation processing to obtain the fused image.

4. A training device for an object recognition model, characterized in that, include: The acquisition module is used to acquire the first monitoring image of the same monitoring point at different times, including the reference image containing the target object and the annotation information; The annotation information is used to characterize the recognition result of the target object in the reference image, and the accuracy of the annotation information is greater than the accuracy threshold; the acquisition module is specifically used to: input the test image including the target object into the current object recognition model to obtain the recognition result of each test image; and use the target recognition result and the test image corresponding to the recognition result with an accuracy greater than the accuracy threshold as the reference image; The target recognition result is the adjusted recognition result in response to the input adjustment operation; The target recognition result and the recognition result with an accuracy greater than the accuracy threshold are used as the annotation information; The generation module is used to generate a fused image of the same monitoring point at different times based on the first monitoring image and the reference image obtained by the acquisition module. The fused image includes the target object and the background of the first monitoring image. The determining module is used to determine the annotation information obtained by the acquiring module as the annotation result of the fused image generated by the generating module; The training module is used to iteratively train the current object recognition model based on the fused image generated by the generation module and the annotation result determined by the determination module until the model converges, thereby obtaining a first target object recognition model; the first target object recognition model is used to identify the target object in the first monitoring image of the same monitoring point; The acquisition module is further configured to: acquire a second monitoring image of the same monitoring point including the object to be identified; input the second monitoring image into the first target object recognition model to obtain an intermediate recognition result of the object to be identified in the second monitoring image; and, in response to an adjustment operation on the intermediate recognition result, acquire the adjusted intermediate recognition result. The adjusted intermediate recognition result is used to characterize whether the object to be identified is the target object; the adjusted intermediate recognition result is determined as the annotation result of the second monitoring image; The training module is further configured to: iteratively train the first target object recognition model based on the second monitoring image and the annotation results of the second monitoring image to obtain a second target object recognition model.

5. The training device according to claim 4, characterized in that, The acquisition module is further configured to: acquire an image of the target object in the reference image; The generation module is specifically used to fuse the first monitoring image and the image of the target object according to a preset image fusion algorithm to obtain the fused image.

6. The training device according to claim 5, characterized in that, The generation module is specifically used for: According to a preset image fusion algorithm, the first monitoring image and the image of the target object are fused to obtain an intermediate image; The intermediate image is subjected to data augmentation processing to obtain the fused image.

7. A computer-readable storage medium, characterized in that, The instruction includes instructions that, when executed on a training device for an object recognition model, cause the training device for the object recognition model to perform the training method as described in any one of claims 1-3.

8. A training device for an object recognition model, characterized in that, include: Memory and processor; the memory and processor are coupled; The memory is used to store computer program code, which includes computer instructions; when the processor executes the computer instructions, the training device for the object recognition model performs the training method for the object recognition model as described in any one of claims 1-3.

Images