A target scene recognition preprocessing method and a target scene recognition method

Abstract

The application discloses a kind of target scene identification pre-processing method, target scene identification method, the method receives the information of each target object type in the target scene to be identified of input;For each target object type, according to the correspondence between each rule of pre-saved rule base and object type, obtain each candidate rule corresponding to the target object type;Receive the rule selection instruction of user, select the target rule corresponding to target scene from each candidate rule obtained;The industry big model of training completion pre-saved and target rule corresponding target scene are saved;Since the target rule of target scene is determined in the method when identifying, and the industry big model corresponding target scene is saved, therefore when different scenes are identified, only the rule corresponding to different scenes needs to be obtained again, so that the identification of different scenes can be realized, and the applicability of scene identification is improved.

Description

Technical Field

[0001] This invention relates to the field of model technology, and in particular to a preprocessing method and a target scene recognition method. Background Technology

[0002] In existing model development, in order to handle the business of scene recognition in the field of computer vision (CV), a model is usually trained for a single business and the trained model is used for processing. That is, the model and the business are coupled together, and the business logic is directly used to train the model, which results in the trained model being able to only handle the corresponding business.

[0003] For example, in existing technologies, to handle the target business of identifying campfire scenes, sample images of campfire scenes are obtained and the original algorithm model is trained to obtain a trained model. The business logic is that if there are people, fire, and people dancing in a circle in the image, it is identified as a campfire scene. However, the trained model can only identify campfire scenes and cannot identify scenes such as wood-fire or camping. To identify wood-fire and camping scenes, the model needs to be retrained. Therefore, the models obtained by existing model development methods have poor adaptability for scene recognition. Summary of the Invention

[0004] This invention provides a preprocessing method, a target scene recognition method, an apparatus, a device, and a medium for target scene recognition, in order to solve the problem of poor adaptability of models obtained by existing model development methods.

[0005] This invention provides a preprocessing method for target scene recognition, the method comprising:

[0006] Receive information about the type of each target object in the target scene to be identified;

[0007] For each target object type, each candidate rule corresponding to that target object type is obtained based on the correspondence between each rule in the pre-saved rule base and the object type.

[0008] Receive the user's rule selection instruction, and select the target rule corresponding to the target scenario from each of the acquired candidate rules;

[0009] Save the pre-trained industry-wide model and the target rules corresponding to the target scenario.

[0010] Furthermore, the training process of the aforementioned industry-wide large model includes:

[0011] Based on a pre-saved database, obtain any first sample image from the database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image, and the sample position information of each object in the first sample image;

[0012] The first sample image is input into the original industry large model, and the second label information of each output first sample image is obtained;

[0013] Based on each first label information and each second label information, the parameter values ​​of each parameter in the original industry big model are adjusted.

[0014] Furthermore, before obtaining any first sample image from the pre-saved database and the first label information corresponding to each first sample image, the method further includes:

[0015] Each first sample image in the database is subjected to data cleaning and data augmentation processing, and the processed first sample image is saved in the database.

[0016] Furthermore, before saving the pre-saved trained industry-wide model and the target rules corresponding to the target scenario, the method further includes:

[0017] For each target object type, the target sample set corresponding to the target object type is determined according to the pre-saved correspondence between object types and sample sets; the industry big model is trained according to any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0018] The step of saving the pre-trained industry-wide model and the target rules corresponding to the target scenario includes:

[0019] Save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0020] Further, the step of training the industry-wide large model based on any second sample image and its corresponding third label information from the acquired target sample set to obtain a target object recognition model corresponding to the target object type includes:

[0021] Obtain any second sample image and its corresponding third label information from the target sample set; input the second sample image into the industry big model; obtain the fourth label information of the second sample image output by the industry big model; wherein the third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image.

[0022] Based on the third and fourth tag information, the parameter values ​​of the industry big model are adjusted to obtain the target object recognition model corresponding to the target object type.

[0023] Furthermore, before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, the method further includes:

[0024] For each target object type, the target object recognition model corresponding to that target object type is compressed using model distillation to obtain the compressed target object recognition model corresponding to that target object type.

[0025] Based on the compressed target object recognition model corresponding to each target object type, the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene is performed.

[0026] Accordingly, the present invention provides a target scene recognition method, the method comprising:

[0027] Receive the input image to be processed and the target scene to be identified;

[0028] Obtain the saved industry-wide model and target rules corresponding to the target scenario;

[0029] The image to be processed is input into the industry big model, and the location information of each target object in the image to be processed and the output of the industry big model are obtained.

[0030] Based on the matching result of whether each target object and its location information conform to the target rule, it is determined whether the image to be processed is the recognition result of the target scene.

[0031] Furthermore, obtaining the saved industry-wide model and target rules corresponding to the target scenario includes:

[0032] Obtain the target object recognition model and target rules corresponding to each target object type stored in the target scene;

[0033] The step of inputting the image to be processed into the industry big model and obtaining each target object and its location information in the image to be processed output by the industry big model includes:

[0034] For each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to that target object type, and the target objects belonging to that target object type and their location information in the output image to be processed are obtained.

[0035] Accordingly, the present invention provides a preprocessing apparatus for target scene recognition, the apparatus comprising:

[0036] The receiving module is used to receive information about each target object type in the target scene to be identified.

[0037] The processing module is used to obtain each candidate rule corresponding to each target object type according to the correspondence between each rule in the pre-saved rule base and the object type for each target object type; receive the user's rule selection instruction, select the target rule corresponding to the target scenario from each obtained candidate rule; and save the pre-saved trained industry large model and the target rule corresponding to the target scenario.

[0038] Furthermore, the device also includes:

[0039] The training module is used to acquire any first sample image from a pre-saved database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image and the sample position information of each object in the first sample image; input the first sample image into the original industry large model, and acquire each second label information of the output first sample image; and adjust the parameter values ​​of each parameter of the original industry large model according to each first label information and each second label information.

[0040] Furthermore, the processing module is also used to perform data cleaning and data augmentation processing on each first sample image in the database before obtaining any first sample image and each first label information corresponding to the first sample image based on the pre-saved database, and to save each processed first sample image in the database.

[0041] Furthermore, the processing module is also used to, before saving the pre-saved trained industry large model and the target rules corresponding to the target scene, determine the target sample set corresponding to each target object type according to the pre-saved correspondence between object types and sample sets; and train the industry large model according to any second sample image and corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0042] The processing module is specifically used to save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0043] Further, the processing module is specifically used to acquire any second sample image and its corresponding third label information from the target sample set, input the second sample image into the industry big model, acquire the fourth label information of the second sample image output by the industry big model, wherein the third label information is used to identify the target object of the target object type in the second sample image, and the sample position information of the target object in the second sample image; and adjust the parameter values ​​of the industry big model according to the third label information and the fourth label information to obtain the target object recognition model corresponding to the target object type.

[0044] Furthermore, the processing module is also used to, before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, compress the target object recognition model corresponding to each target object type using model distillation to obtain a compressed target object recognition model corresponding to that target object type; and based on the compressed target object recognition model corresponding to each target object type, perform the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0045] Accordingly, the present invention provides a target scene recognition device, the device comprising:

[0046] The receiving module is used to receive the input image to be processed and the target scene to be identified;

[0047] The acquisition module is used to acquire the industry big model and target rules corresponding to the target scene; input the image to be processed into the industry big model, and acquire each target object and the position information of each target object in the image to be processed output by the industry big model;

[0048] The recognition module is used to determine whether the image to be processed is the recognition result of the image of the target scene based on whether the location information of each target object and the target object conforms to the matching result of the target rule.

[0049] Furthermore, the acquisition module is specifically used to acquire the target object recognition model and target rules corresponding to each target object type stored in the target scene; for each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to the target object type, and the target object belonging to the target object type and the position information of the target object in the output image to be processed are acquired.

[0050] Accordingly, the present invention provides an electronic device, the electronic device including a processor and a memory, the memory being used to store program instructions, and the processor being used to execute the computer program stored in the memory to implement the steps of any of the above-described preprocessing methods for target scene recognition, or to implement the steps of any of the above-described target scene recognition methods.

[0051] Accordingly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described preprocessing methods for target scene recognition, or implements the steps of any of the above-described target scene recognition methods.

[0052] This invention provides a preprocessing method, a target scene recognition method, an apparatus, a device, and a medium for target scene recognition. The method receives information about each target object type in an input target scene to be recognized; for each target object type, it obtains each candidate rule corresponding to that target object type based on a pre-saved rule base's correspondence between each rule and object type; it receives a user's rule selection instruction and selects the target rule corresponding to the target scene from the obtained candidate rules; and it saves a pre-saved, trained industry-wide model and the target rule corresponding to the target scene. Since this method determines the target rule for the target scene and saves it corresponding to the industry-wide model when recognizing the target scene, it only needs to re-obtain the rules corresponding to different scenes when recognizing different scenes, thus enabling the recognition of different scenes and improving the applicability of scene recognition. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A schematic diagram of a model category provided for existing technology;

[0055] Figure 2 A schematic diagram of an environment architecture type for a model provided by existing technology;

[0056] Figure 3 A schematic diagram illustrating the development of a model for existing technologies;

[0057] Figure 4 A schematic diagram illustrating the process of a preprocessing method for target scene recognition provided in an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram illustrating the process of a target scene recognition method provided in an embodiment of the present invention;

[0059] Figure 6 A schematic diagram illustrating target scene recognition provided in an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of the structure of a preprocessing device for target scene recognition provided in an embodiment of the present invention;

[0061] Figure 8 This is a schematic diagram of the structure of a target scene recognition device provided in an embodiment of the present invention;

[0062] Figure 9 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0064] Model development serves business needs, efficiently solving business difficulties and pain points. However, model development is not an easy task; ensuring each step is reasonable and effective is crucial for successful model development. Key aspects include the rationality of model positioning, the reliability of data quality, the accuracy of model output, and the stability of model performance.

[0065] In practical work, the types of models used include various categories. Figure 1 A schematic diagram of a model category provided for existing technology, such as Figure 1 As shown, the model categories include four types: image classification model, object detection model, key point detection model, and semantic segmentation model. In terms of the ease of extending the generality of the model, the image classification model is easier than the object detection model, the object detection model is easier than the key point detection model, and the key point detection model is easier than the semantic segmentation model.

[0066] Figure 2 A schematic diagram of an environment architecture type for a model provided by existing technology, such as... Figure 2 As shown, the model's environment architecture mainly includes deep learning frameworks (Torch), computer vision frameworks (OpenCV), open-source artificial neural network libraries (Keras), and deep learning platforms (Paddle). The model's environment architecture is mainly related to the ease of model building and deployment. Currently, Torch and OpenCV are mainly used, while Paddle is used for edge computing in China.

[0067] Existing technologies involve strongly binding the model to business logic, adding post-processing and business logic directly after the model. Figure 3 A schematic diagram for the development of a model for existing technologies, such as Figure 3 As shown, the model development process includes the following steps: S301: Input training data into the model; S302: Process the training data based on the model, post-processing, and business logic to obtain the output content; S303: Receive business operation instructions and modify them.

[0068] To improve the applicability of scene recognition, embodiments of the present invention provide a preprocessing method, a target scene recognition method, an apparatus, a device, and a medium for target scene recognition.

[0069] Example 1:

[0070] Figure 4 This is a schematic diagram of a preprocessing method for target scene recognition provided in an embodiment of the present invention. The process includes the following steps:

[0071] S401: Receive information about the type of each target object in the target scene to be identified.

[0072] To improve the applicability of scene recognition, the preprocessing method for target scene recognition provided in this embodiment of the invention is applied to electronic devices, wherein the electronic devices may be smart terminals such as host computers, tablet computers, and mobile terminals, or servers; the servers may be local servers or cloud servers; specifically, this embodiment of the invention does not impose any limitations on this.

[0073] To enable preprocessing for target scene recognition, the electronic device receives information on each target object type in the target scene to be recognized. The target scene to be recognized refers to a specific state composed of target objects of different object types in a natural or social environment. Different object types include humans, objects, etc. The target scene can be a campfire party scene, a work scene, a gas meter maintenance scene, etc. The information on each target object type refers to the identification information that identifies each target object type. The information on each target object type can be the name of the target object type, a pre-saved number, etc.

[0074] Specifically, the electronic device may receive information on each target object type in the target scene to be identified via a touchscreen, a keyboard, a voice input device, or other means, or it may receive such information via other devices connected to the electronic device.

[0075] S402: For each target object type, obtain each candidate rule corresponding to that target object type according to the correspondence between each rule in the pre-saved rule base and the object type.

[0076] After acquiring information about each target object type, in order to determine the target rules for target scene recognition, the electronic device pre-stores a rule library. This rule library stores the correspondence between each object type and each rule. Specifically, each rule corresponds to the object type information. Each rule stored in the rule library is pre-written and saved by business engineers. Therefore, the structure and style of each rule in the rule library are consistent, which facilitates subsequent expansion and modification.

[0077] For each target object type, the electronic device determines each rule corresponding to the target object type based on the information of the target object type and the pre-saved correspondence, and determines each rule corresponding to the target object type as a candidate rule.

[0078] S403: Receive the user's rule selection instruction and select the target rule corresponding to the target scenario from each of the acquired candidate rules.

[0079] Based on each candidate rule obtained, the electronic device receives the user's rule selection instruction, selects the rule corresponding to the rule identification information from each candidate rule according to the rule identification information carried in the rule selection instruction, and determines it as the target rule corresponding to the target scenario.

[0080] Specifically, the electronic device may receive the user's rule selection instruction through the display screen, or through pressing, sliding, or clicking operations at preset positions on the display screen, and receive the user's input rule selection instruction, or receive the user's rule selection instruction sent by other devices connected to the terminal device.

[0081] S404: Save the pre-trained industry-wide model and the target rules corresponding to the target scenario.

[0082] After determining the target rule, the electronic device saves the pre-trained industry big model and the target scene corresponding to the target rule. The pre-trained industry big model is a model that can recognize multiple object types in the target industry. This industry big model is a model that has been pre-trained in an unsupervised manner based on the database of the target industry.

[0083] Specifically, the target rule runs on the cognitive intelligence platform, the pre-trained industry-wide model runs on the perceptual intelligence platform, and the perceptual intelligence platform and the cognitive intelligence platform store the corresponding target scenes, which are then used to recognize the target scenes.

[0084] In this embodiment of the invention, the method receives information about each target object type in the input target scene to be identified; for each target object type, it obtains each candidate rule corresponding to that target object type according to the correspondence between each rule in the pre-saved rule base and the object type; it receives a rule selection instruction from the user and selects the target rule corresponding to the target scene from each obtained candidate rule; it saves the pre-saved trained industry large model and the target rule corresponding to the target scene; since this method determines the target rule of the target scene when identifying the target scene and saves it with the industry large model corresponding to the target scene, when identifying different scenes, it is only necessary to re-obtain the rules corresponding to different scenes, thereby realizing the identification of different scenes and improving the applicability of scene identification.

[0085] Because the model and rules are separated in this embodiment of the invention, algorithm engineers can focus more on adjusting the data and model, and business engineers can focus more on the rules; parallel development can be carried out on both sides when necessary, which improves work efficiency; unified input and output interfaces are conducive to the standardization of rule code and improve code development efficiency.

[0086] Example 2:

[0087] To obtain a fully trained industry-specific large-scale model, based on the above embodiments, the training process of the industry-specific large-scale model in this embodiment of the invention includes:

[0088] Based on a pre-saved database, obtain any first sample image from the database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image, and the sample position information of each object in the first sample image;

[0089] The first sample image is input into the original industry large model, and the second label information of each output first sample image is obtained;

[0090] Based on each first label information and each second label information, the parameter values ​​of each parameter in the original industry big model are adjusted.

[0091] To enable the training of a large industry model, this embodiment of the invention stores a training database. This database is an industry database for the target industry. The first sample image in the database includes scene images of various scenarios in the target industry. The first label information of the first sample image is predefined and labeled manually. The first label information is used to identify the object type of each object in the first sample image and the sample position information of each object in the first sample image.

[0092] In this embodiment of the invention, after obtaining any first sample image and its first label information from the database, the first sample image is input into the original industry big data model, which outputs the second label information of the first sample image. The second label information identifies the object type of each object in the first sample image and the identification location information of each object within the first sample image.

[0093] After determining the second label information of the first sample image based on the original industry big model, the original industry big model is trained based on the second label information and the first label information of the first sample image to adjust the parameter values ​​of various parameters of the original industry big model.

[0094] The above operation is performed on each first sample image in the database used to train the original industry large model. When a preset condition is met, the trained industry large model is obtained. This preset condition may be that the number of first sample images in the database whose second label information matches the first label information obtained after training the original industry large model is greater than a set number; or that the number of iterations for training the original industry large model reaches a set maximum number of iterations, etc. Specifically, this embodiment of the invention does not impose limitations on these aspects.

[0095] As one possible implementation, when training the original industry large model, the first sample images in the database can be divided into first training sample images and first test sample images. The original industry large model is first trained based on the first training sample images, and then the reliability of the trained industry large model is tested based on the first test sample images.

[0096] Existing technologies require training multiple models for multiple scenarios, but the training data in the sample sets of different models suffers from data silos, resulting in low data utilization. Furthermore, each model requires labeling the training data, leading to high time and manpower costs for manual labeling. In contrast, this invention employs a unified, pre-trained industry-specific model, which can be used as a pre-training model for object recognition. Because it is built with an industry-specific database, it has good adaptability to industry scenarios, reducing the time spent on initial model research and validation. Moreover, since the industry-specific model incorporates the identification of specific needs, the algorithm task can be controlled from a macro perspective.

[0097] To improve the accuracy and efficiency of training large industry models, in this embodiment of the invention, before obtaining any first sample image from the pre-saved database and the first label information corresponding to each first sample image, the method further includes:

[0098] Each first sample image in the database is subjected to data cleaning and data augmentation processing, and the processed first sample image is saved in the database.

[0099] To improve the accuracy and efficiency of training large industry models, in this embodiment of the invention, the electronic device performs data cleaning and data augmentation processing on each first sample image in the database. Data cleaning refers to deleting duplicate or damaged first sample images from the database; data augmentation refers to enhancing the brightness and clarity of the first sample images in the database, thereby improving the accuracy of training large industry models.

[0100] After cleaning and augmenting each first sample image, the processed first sample images are saved in the database, and the original industry large model is trained based on each processed first sample image in the database.

[0101] Because the database in this embodiment integrates training data from sample sets used in existing technologies for identifying different scenarios, it eliminates data silos, improves data utilization and reuse, and reduces additional manual labor and time costs for data requiring manual annotation. Since the quality and quantity of data are the cornerstones of a model's success, this embodiment employs data cleaning and data augmentation processes, thereby improving the quality of industry data, increasing the training accuracy of large industry models, shortening model training time, and reducing manpower and time costs.

[0102] Example 3:

[0103] To improve the accuracy of scene recognition, based on the above embodiments, in this embodiment of the invention, before saving the pre-saved trained industry large model and the target rules corresponding to the target scene, the method further includes:

[0104] For each target object type, the target sample set corresponding to the target object type is determined according to the pre-saved correspondence between object types and sample sets; the industry big model is trained according to any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0105] The step of saving the pre-trained industry-wide model and the target rules corresponding to the target scenario includes:

[0106] Save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0107] Since the industry-wide model means that every target in the industry can be identified, it can also lead to poor object recognition accuracy. In order to improve the accuracy of scene recognition, the electronic device pre-stores a sample set corresponding to each object type. For each target object type in the target scene, the electronic device determines the target sample set corresponding to that target object type.

[0108] Based on the determined target sample set, any second sample image and its corresponding third label information are obtained from the target sample set. The third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image. Based on the obtained second sample image and its corresponding third label information, the electronic device performs subsequent training on the industry large model to obtain the target object recognition model corresponding to the target object type.

[0109] As one possible implementation, in this embodiment of the invention, the electronic device can also receive parameter adjustment instructions from the user to adjust the parameters of the industry-wide model. Specifically, it can use fine-tuning to adjust the parameters of the industry-wide model to obtain a target object recognition model corresponding to each target object type.

[0110] After determining the target object recognition model corresponding to each target object type, the electronic device saves the target object recognition model and the target scene corresponding to each target rule. Specifically, the target rule runs on the cognitive intelligence platform, and the target object recognition model corresponding to each target object type runs on the perceptual intelligence platform. The perceptual intelligence platform and the cognitive intelligence platform together form a perceptual-cognitive separation model for target scene recognition, thereby realizing the recognition of the target scene based on the perceptual-cognitive separation model.

[0111] Example 4:

[0112] To obtain a target object recognition model, based on the above embodiments, in this embodiment of the invention, the step of training the industry-wide model based on any second sample image and corresponding third label information from the acquired target sample set to obtain a target object recognition model corresponding to the target object type includes:

[0113] Obtain any second sample image and its corresponding third label information from the target sample set; input the second sample image into the industry big model; obtain the fourth label information of the second sample image output by the industry big model; wherein the third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image.

[0114] Based on the third and fourth tag information, the parameter values ​​of the industry big model are adjusted to obtain the target object recognition model corresponding to the target object type.

[0115] In order to train the target object recognition model corresponding to the target object type, the present invention stores a target sample set corresponding to the target object type. The second sample image in the target sample set includes scene images of various scenes. The third label information of the second sample image is manually pre-annotated. The third label information is used to identify the target object belonging to the target object type in the first sample image and the sample position information of the target object in the second sample image.

[0116] In this embodiment of the invention, after obtaining any second sample image from the target sample set and its third label information, the second sample image is input into an industry-wide model, which outputs the fourth label information of the second sample image. The fourth label information identifies the target object in the second sample image as belonging to the target object type, and also indicates the target object's location within the second sample image.

[0117] After determining the fourth label information of the second sample image based on the industry big model, the industry big model is trained based on the third label information and the fourth label information of the second sample image to adjust the parameter values ​​of various parameters of the industry big model, thereby obtaining the target object recognition model corresponding to the target object type.

[0118] The above operation is performed on each of the target images in the target sample set used to train the industry-wide model. When a preset condition is met, a target object recognition model corresponding to the target object type is obtained after training. This preset condition may be that the number of second sample images in the database whose fourth label information matches the third label information after training with the industry-wide model is greater than a set number; or that the number of iterations for training the industry-wide model reaches a set maximum number of iterations, etc. Specifically, this embodiment of the invention does not limit this.

[0119] As one possible implementation, when training the industry-wide model, the second sample images in the target sample set can be divided into second training sample images and second test sample images. The industry-wide model is first trained based on the second training sample images, and then the reliability of the target object recognition model corresponding to the trained target object type is tested based on the second test sample images.

[0120] Example 5:

[0121] To reduce memory resource consumption, based on the above embodiments, in this embodiment of the invention, before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, the method further includes:

[0122] For each target object type, the target object recognition model corresponding to that target object type is compressed using model distillation to obtain the compressed target object recognition model corresponding to that target object type.

[0123] Based on the compressed target object recognition model corresponding to each target object type, the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene is performed.

[0124] To reduce memory resource consumption, in this embodiment of the invention, the electronic device also compresses the target object recognition model for each target object type. Specifically, it uses model distillation to compress the target object recognition model for that target object type, thereby obtaining the compressed target object recognition model corresponding to that target object type.

[0125] Based on the compressed target object recognition model corresponding to each target object type, the electronic device performs the subsequent steps of saving the target object recognition model and target rule corresponding to each target object type and the target scene, so as to realize the recognition of the target scene.

[0126] Example 6:

[0127] Figure 5 This is a schematic diagram of a target scene recognition method provided in an embodiment of the present invention. The process includes the following steps:

[0128] S501: Receive the input image to be processed and the target scene to be identified.

[0129] To achieve the recognition of target scenes, the target scene recognition method provided in this embodiment of the invention is applied to an electronic device, wherein the electronic device may be a host, tablet computer, mobile terminal or other smart terminal, or a server; the server may be a local server or a cloud server; specifically, this embodiment of the invention does not limit this.

[0130] The electronic device receives an input image to be processed and a target scene to be identified. The image to be processed is used to determine whether the scene in the image is the target scene. The electronic device may receive the image to be processed from other devices connected to it, or it may receive the image to be processed after being captured by the camera of the electronic device. The electronic device may receive the input target scene through a touch screen, or it may receive the input target scene through a voice acquisition device.

[0131] S502: Obtain the industry big model and target rules corresponding to the target scene; input the image to be processed into the industry big model, and obtain each target object and the position information of each target object in the image to be processed output by the industry big model.

[0132] Based on the industry-wide model and target rules stored in the preprocessing stage of target scene recognition, the electronic device obtains the stored industry-wide model and target rules, wherein the industry-wide model is pre-trained to recognize each target in the image to be processed.

[0133] The electronic device inputs the received image to be processed into the industry big model, and obtains each target object in the image to be processed from the output of the industry big model, as well as the position information of each target object in the image to be processed.

[0134] Specifically, the electronic device inputs the image to be processed into an industry-wide model running on a perceptual intelligence platform. The industry-wide model performs preprocessing, recognition processing, and postprocessing on the image to be processed, and outputs each target object in the image to be processed, as well as the position information of each target object in the image to be processed.

[0135] S503: Based on whether the location information of each target object and the target object match the target rule, determine whether the image to be processed is the recognition result of the image of the target scene.

[0136] Based on the target object, its location information, and the target rules corresponding to the target scene output by the industry big data model, it is determined whether each target object and its location information conform to the target rules. If the matching result is that each target object and its location information conforms to the target rules, then the recognition result is determined to be an image of the target scene; if the matching result is that each target object and its location information does not conform to the target rules, then the recognition result is determined to be an image of the target scene.

[0137] Example 7:

[0138] To improve the accuracy of target scene recognition, based on the above embodiments, in this embodiment of the invention, obtaining the stored industry-wide model and target rules corresponding to the target scene includes:

[0139] Obtain the target object recognition model and target rules corresponding to each target object type stored in the target scene;

[0140] The step of inputting the image to be processed into the industry big model and obtaining each target object and its location information in the image to be processed output by the industry big model includes:

[0141] For each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to that target object type, and the target objects belonging to that target object type and their location information in the output image to be processed are obtained.

[0142] To improve the accuracy of target scene recognition, in this embodiment of the invention, the electronic device acquires the target object recognition model and target rules corresponding to each target object type stored in the target scene. For each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to the target object type, and the target object belonging to the target object type in the image to be processed and the position information of the target object in the image to be processed are obtained from the target object recognition model corresponding to the target object type.

[0143] The following describes a target scene recognition method according to a specific embodiment of the present invention. Figure 6 This is a schematic diagram of target scene recognition provided in an embodiment of the present invention, such as... Figure 6 As shown, the electronic device receives the input image to be processed and inputs the image to be processed into the perceptual intelligence platform. The target object recognition model corresponding to each target object type, which is run by the perceptual intelligence platform, identifies the objects in the image to be processed and determines each target object and the location information of each target object in the output image to be processed. The target object recognition model for each target object type can be at least one of the following: target detection model, vector retrieval model, and semantic segmentation model.

[0144] Based on the target rules obtained from the business rule base and running on the cognitive intelligence platform, each target object in the image to be processed and its location information are input into the cognitive intelligence platform to determine whether each target object and its location information conform to the matching result of the target rules, thereby determining whether the image to be processed is the recognition result of the target scene.

[0145] Example 8:

[0146] Figure 7 This is a schematic diagram of the structure of a preprocessing device for target scene recognition provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the device includes:

[0147] The receiving module 701 is used to receive information about each target object type in the target scene to be identified.

[0148] The processing module 702 is used to obtain each candidate rule corresponding to each target object type according to the correspondence between each rule and the object type in the pre-saved rule base; receive the user's rule selection instruction, select the target rule corresponding to the target scenario from each obtained candidate rule; and save the pre-saved trained industry large model and the target rule corresponding to the target scenario.

[0149] Furthermore, the device also includes:

[0150] Training module 703 is used to obtain any first sample image from the pre-saved database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image and the sample position information of each object in the first sample image; input the first sample image into the original industry large model, and obtain each second label information of the output first sample image; and adjust the parameter values ​​of each parameter of the original industry large model according to each first label information and each second label information.

[0151] Furthermore, the processing module 702 is also used to perform data cleaning and data augmentation processing on each first sample image in the database before obtaining any first sample image in the database based on the pre-saved database and each first label information corresponding to the first sample image, and to save each processed first sample image in the database.

[0152] Furthermore, the processing module 702 is also used to, before saving the pre-saved trained industry large model and the target rules corresponding to the target scene, determine the target sample set corresponding to each target object type according to the pre-saved correspondence between object types and sample sets; and train the industry large model according to any second sample image and corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0153] The processing module 702 is specifically used to save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0154] Further, the processing module 702 is specifically used to acquire any second sample image and its corresponding third label information from the target sample set, input the second sample image into the industry big model, acquire the fourth label information of the second sample image output by the industry big model, wherein the third label information is used to identify the target object of the target object type in the second sample image, and the sample position information of the target object in the second sample image; and adjust the parameter values ​​of the industry big model according to the third label information and the fourth label information to obtain the target object recognition model corresponding to the target object type.

[0155] Furthermore, the processing module 702 is also used to, before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, compress the target object recognition model corresponding to each target object type using model distillation to obtain a compressed target object recognition model corresponding to that target object type; and based on the compressed target object recognition model corresponding to each target object type, perform the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0156] Example 9:

[0157] Figure 8 This is a schematic diagram of the structure of a target scene recognition device provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the device includes:

[0158] The receiving module 801 is used to receive the input image to be processed and the target scene to be identified;

[0159] The acquisition module 802 is used to acquire the industry big model and target rules corresponding to the target scene; input the image to be processed into the industry big model, and acquire each target object and the position information of each target object in the image to be processed output by the industry big model;

[0160] The recognition module 803 is used to determine whether the image to be processed is the recognition result of the image of the target scene based on whether the location information of each target object and the target object conforms to the matching result of the target rule.

[0161] Furthermore, the acquisition module 802 is specifically used to acquire the target object recognition model and target rules corresponding to each target object type stored in the target scene; for each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to the target object type, and the target object belonging to the target object type and the position information of the target object in the output image to be processed are acquired.

[0162] Example 10:

[0163] Figure 9 This is a schematic diagram of an electronic device structure provided by an embodiment of the present invention. Based on the above embodiments, this invention also provides an electronic device, such as... Figure 9 As shown, it includes: processor 901, communication interface 902, memory 903 and communication bus 904, wherein processor 901, communication interface 902 and memory 903 communicate with each other through communication bus 904.

[0164] The memory 903 stores a computer program, which, when executed by the processor 901, causes the processor 901 to perform the following steps:

[0165] Receive information about the type of each target object in the target scene to be identified;

[0166] For each target object type, each candidate rule corresponding to that target object type is obtained based on the correspondence between each rule in the pre-saved rule base and the object type.

[0167] Receive the user's rule selection instruction, and select the target rule corresponding to the target scenario from each of the acquired candidate rules;

[0168] Save the pre-trained industry-wide model and the target rules corresponding to the target scenario.

[0169] Furthermore, the processor 901 is specifically used in the training process of the large industry model, including:

[0170] Based on a pre-saved database, obtain any first sample image from the database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image, and the sample position information of each object in the first sample image;

[0171] The first sample image is input into the original industry large model, and the second label information of each output first sample image is obtained;

[0172] Based on each first label information and each second label information, the parameter values ​​of each parameter in the original industry big model are adjusted.

[0173] Furthermore, before the processor 901 obtains any first sample image from the pre-saved database and the first label information corresponding to each first sample image, the method further includes:

[0174] Each first sample image in the database is subjected to data cleaning and data augmentation processing, and the processed first sample image is saved in the database.

[0175] Furthermore, before the processor 901 saves the pre-saved trained industry-wide model and the target rule corresponding to the target scene, the method further includes:

[0176] For each target object type, the target sample set corresponding to the target object type is determined according to the pre-saved correspondence between object types and sample sets; the industry big model is trained according to any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0177] The step of saving the pre-trained industry-wide model and the target rules corresponding to the target scenario includes:

[0178] Save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0179] Further, the processor 901 is specifically used to train the industry-wide large model based on any second sample image and corresponding third label information in the acquired target sample set to obtain a target object recognition model corresponding to the target object type, including:

[0180] Obtain any second sample image and its corresponding third label information from the target sample set; input the second sample image into the industry big model; obtain the fourth label information of the second sample image output by the industry big model; wherein the third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image.

[0181] Based on the third and fourth tag information, the parameter values ​​of the industry big model are adjusted to obtain the target object recognition model corresponding to the target object type.

[0182] Furthermore, before the processor 901 saves the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, the method further includes:

[0183] For each target object type, the target object recognition model corresponding to that target object type is compressed using model distillation to obtain the compressed target object recognition model corresponding to that target object type.

[0184] Based on the compressed target object recognition model corresponding to each target object type, the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene is performed.

[0185] Or the processor 901 may perform the following steps:

[0186] Receive the input image to be processed and the target scene to be identified;

[0187] Obtain the saved industry-wide model and target rules corresponding to the target scenario;

[0188] The image to be processed is input into the industry big model, and the location information of each target object in the image to be processed and the output of the industry big model are obtained.

[0189] Based on the matching result of whether each target object and its location information conform to the target rule, it is determined whether the image to be processed is the recognition result of the target scene.

[0190] Furthermore, the processor 901 is specifically used for obtaining the saved industry-wide model and target rules corresponding to the target scene, including:

[0191] Obtain the target object recognition model and target rules corresponding to each target object type stored in the target scene;

[0192] The step of inputting the image to be processed into the industry big model and obtaining each target object and its location information in the image to be processed output by the industry big model includes:

[0193] For each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to that target object type, and the target objects belonging to that target object type and their location information in the output image to be processed are obtained.

[0194] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0195] The communication interface 902 is used for communication between the above-mentioned electronic device and other devices.

[0196] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0197] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0198] Example 11:

[0199] Based on the above embodiments, this invention also provides a computer-readable storage medium storing a computer program executable by a processor. When the program is run on the processor, the processor executes the following steps:

[0200] Receive information about the type of each target object in the target scene to be identified;

[0201] For each target object type, each candidate rule corresponding to that target object type is obtained based on the correspondence between each rule in the pre-saved rule base and the object type.

[0202] Receive the user's rule selection instruction, and select the target rule corresponding to the target scenario from each of the acquired candidate rules;

[0203] Save the pre-trained industry-wide model and the target rules corresponding to the target scenario.

[0204] Furthermore, the training process of the aforementioned industry-wide large model includes:

[0205] Based on a pre-saved database, obtain any first sample image from the database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image, and the sample position information of each object in the first sample image;

[0206] The first sample image is input into the original industry large model, and the second label information of each output first sample image is obtained;

[0207] Based on each first label information and each second label information, the parameter values ​​of each parameter in the original industry big model are adjusted.

[0208] Furthermore, before obtaining any first sample image from the pre-saved database and the first label information corresponding to each first sample image, the method further includes:

[0209] Each first sample image in the database is subjected to data cleaning and data augmentation processing, and the processed first sample image is saved in the database.

[0210] Furthermore, before saving the pre-saved trained industry-wide model and the target rules corresponding to the target scenario, the method further includes:

[0211] For each target object type, the target sample set corresponding to the target object type is determined according to the pre-saved correspondence between object types and sample sets; the industry big model is trained according to any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type.

[0212] The step of saving the pre-trained industry-wide model and the target rules corresponding to the target scenario includes:

[0213] Save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

[0214] Further, the step of training the industry-wide large model based on any second sample image and its corresponding third label information from the acquired target sample set to obtain a target object recognition model corresponding to the target object type includes:

[0215] Obtain any second sample image and its corresponding third label information from the target sample set; input the second sample image into the industry big model; obtain the fourth label information of the second sample image output by the industry big model; wherein the third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image.

[0216] Based on the third and fourth tag information, the parameter values ​​of the industry big model are adjusted to obtain the target object recognition model corresponding to the target object type.

[0217] Furthermore, before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, the method further includes:

[0218] For each target object type, the target object recognition model corresponding to that target object type is compressed using model distillation to obtain the compressed target object recognition model corresponding to that target object type.

[0219] Based on the compressed target object recognition model corresponding to each target object type, the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene is performed.

[0220] Or cause the processor to perform the following steps:

[0221] Receive the input image to be processed and the target scene to be identified;

[0222] Obtain the saved industry-wide model and target rules corresponding to the target scenario;

[0223] The image to be processed is input into the industry big model, and the location information of each target object in the image to be processed and the output of the industry big model are obtained.

[0224] Based on the matching result of whether each target object and its location information conform to the target rule, it is determined whether the image to be processed is the recognition result of the target scene.

[0225] Furthermore, obtaining the saved industry-wide model and target rules corresponding to the target scenario includes:

[0226] Obtain the target object recognition model and target rules corresponding to each target object type stored in the target scene;

[0227] The step of inputting the image to be processed into the industry big model and obtaining each target object and its location information in the image to be processed output by the industry big model includes:

[0228] For each target object type in the target scene, the image to be processed is input into the target object recognition model corresponding to that target object type, and the target objects belonging to that target object type and their location information in the output image to be processed are obtained.

[0229] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0231] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0232] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0233] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A preprocessing method for target scene recognition, characterized in that, The method includes: Receive information about the type of each target object in the target scene to be identified; For each target object type, each candidate rule corresponding to that target object type is obtained based on the correspondence between each rule in the pre-saved rule base and the object type. Receive the user's rule selection instruction, and select the target rule corresponding to the target scenario from each of the acquired candidate rules; For each target object type, the target sample set corresponding to the target object type is determined according to the pre-saved correspondence between object types and sample sets; the industry large model is trained according to any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to the target object type. Save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

2. The method according to claim 1, characterized in that, The training process of the aforementioned industry-wide large model includes: Based on a pre-saved database, obtain any first sample image from the database, and each first label information corresponding to the first sample image, wherein each first label information identifies the object type of each object in the first sample image, and the sample position information of each object in the first sample image; The first sample image is input into the original industry large model, and the second label information of each output first sample image is obtained; Based on each first label information and each second label information, the parameter values ​​of each parameter in the original industry big model are adjusted.

3. The method according to claim 2, characterized in that, Before obtaining any first sample image from the pre-saved database and the first label information corresponding to each first sample image, the method further includes: Each first sample image in the database is subjected to data cleaning and data augmentation processing, and the processed first sample image is saved in the database.

4. The method according to claim 1, characterized in that, The step of training the industry-wide model based on any second sample image and its corresponding third label information from the acquired target sample set to obtain a target object recognition model corresponding to the target object type includes: Obtain any second sample image and its corresponding third label information from the target sample set; input the second sample image into the industry big model; obtain the fourth label information of the second sample image output by the industry big model; wherein the third label information is used to identify the target object of the target object type in the second sample image and the sample position information of the target object in the second sample image. Based on the third and fourth tag information, the parameter values ​​of the industry big model are adjusted to obtain the target object recognition model corresponding to the target object type.

5. The method according to claim 1, characterized in that, Before saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene, the method further includes: For each target object type, the target object recognition model corresponding to that target object type is compressed using model distillation to obtain the compressed target object recognition model corresponding to that target object type. Based on the compressed target object recognition model corresponding to each target object type, the subsequent step of saving the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene is performed.

6. A preprocessing device for target scene recognition, characterized in that, The device includes: The receiving module is used to receive information about each target object type in the target scene to be identified. The processing module is configured to: for each target object type, obtain each candidate rule corresponding to that target object type based on the pre-saved correspondence between each rule in the rule base and the object type; receive a rule selection instruction from the user and select the target rule corresponding to the target scene from each obtained candidate rule; for each target object type, determine the target sample set corresponding to that target object type based on the pre-saved correspondence between the object type and the sample set; train the industry large model based on any second sample image and the corresponding third label information in the obtained target sample set to obtain the target object recognition model corresponding to that target object type; and save the target object recognition model corresponding to each target object type and the target rule corresponding to the target scene.

7. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program, which, when executed by the processor, causes the processor to execute the computer program stored in the memory to implement the preprocessing method for target scene recognition as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program executable by a processor, which, when run on the processor, causes the processor to implement the preprocessing method for target scene recognition as described in any one of claims 1-5.

Images