A data classification method, computer and readable storage medium

Abstract

The embodiment of the present application discloses a data classification method, a computer and a readable storage medium, which relates to the field of artificial intelligence. The method includes: obtaining a visual identifier corresponding to an original image, identifying the detection position coordinates used to extract an object of interest in the original image, obtaining a target detection image containing the object of interest from the original image based on the detection position coordinates, and determining the detection position information used to extract the object of interest according to the visual identifier and the detection position coordinates; the visual identifier is used to represent the detection vision corresponding to the original image in the object of interest; obtaining image features of the target detection image, performing a scale transformation on the detection position information, and obtaining position features of the target detection image; performing feature splicing on the image features and the position features to obtain fused features, and performing classification processing on the fused features to obtain the image category to which the target detection image belongs. The present application can improve the accuracy of data recognition and classification.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data classification method, a computer and a readable storage medium. Background Art

[0002] With the advent of information and intelligent society, industrial product production has gradually moved towards intelligent production, which has greatly improved productivity. However, with the large-scale liberation of labor, how to improve product quality and yield rate, thereby reducing raw material consumption and labor cost investment, has become a core issue faced by factories in the process of digital and intelligent transformation. Therefore, the detection of products or parts is extremely important, and defect detection has become a very important application in industry. Among them, in the field of industrial defect detection, manual inspection is generally used, but manual inspection is costly and easily affected by the proficiency of workers, resulting in large differences in detection accuracy and efficiency; or, the method of first detecting (or segmenting) and then performing secondary classification is used to identify industrial defects, that is, directly using target detection models to identify industrial defects, resulting in low accuracy of recognition results. Summary of the invention

[0003] The embodiments of the present application provide a data classification method, a computer, and a readable storage medium, which can improve the accuracy of data identification and classification.

[0004] On the one hand, an embodiment of the present application provides a data classification method, the method comprising:

[0005] Obtaining a visual identifier corresponding to the original image, identifying the detection position coordinates for extracting the object of interest in the original image, obtaining a target detection image containing the object of interest from the original image based on the detection position coordinates, and determining the detection position information corresponding to the original image based on the visual identifier and the detection position coordinates; the visual identifier is used to indicate the visual angle of the object of interest in the object corresponding to the original image;

[0006] Acquire the image features of the target detection image, perform scale transformation on the detection position information, and obtain the position features of the target detection image;

[0007] The image features and position features are concatenated to obtain fused features. The key features in the image features are determined based on the position features. The fused features are classified based on the key features to obtain the image category to which the target detection image belongs.

[0008] The method of identifying the detection position coordinates for extracting the object of interest in the original image and obtaining the target detection image containing the object of interest from the original image based on the detection position coordinates includes:

[0009] Acquire an original image, perform object detection on the original image, obtain a first predicted bounding box containing the object of interest, and determine the detection position coordinates according to first position information of the first predicted bounding box;

[0010] A second predicted border is determined according to the detection position coordinates, and an area indicated by the second predicted border in the original image is determined as the target detection image.

[0011] Wherein, determining the detection position coordinates according to the first position information of the first predicted frame includes:

[0012] Determining a border width and a border height of the first predicted border according to the first position information of the first predicted border;

[0013] The first predicted border is resized based on the border width and the border height, and the detection position coordinates are determined according to the adjusted first predicted border.

[0014] The first predicted border is resized based on the border width and the border height, and the detection position coordinates are determined according to the adjusted first predicted border, including:

[0015] According to the difference between the border width and the border height and the first position information, the first predicted border is resized to determine the second position information and the adjusted width and height of the adjusted first predicted border;

[0016] Obtain a frame expansion coefficient, transform the size of the adjustment width and the adjustment height based on the frame expansion coefficient, and determine the detection position coordinates according to the second position information and the transformed adjustment width and adjustment height.

[0017] Wherein, obtaining the image features of the target detection image includes:

[0018] The target detection image is input into the convolutional neural network in the data classification model, and features of the target detection image are extracted based on the convolutional layer in the convolutional neural network to obtain image features of the target detection image.

[0019] Among them, the detection position information is scaled to obtain the position features of the target detection image, including:

[0020] Obtaining the image width and image height of the target detection image, and normalizing the detection position coordinates based on the image width and image height to obtain normalized position coordinates;

[0021] Obtaining a visual total number, and normalizing the visual identifier based on the visual total number to obtain a normalized visual identifier;

[0022] According to the normalized position coordinates and the normalized visual identifier, the perceptual input features are generated, and the multi-layer perceptron in the data classification model is used to scale the perceptual input features to obtain the position features of the target detection image.

[0023] Among them, the image features and position features are spliced ​​to obtain fusion features, including:

[0024] Normalizing the image features to obtain normalized image features, and normalizing the position features to obtain normalized position features;

[0025] The normalized image features and normalized position features are concatenated to obtain fused features.

[0026] Among them, the fusion features are classified based on the key features to obtain the image category to which the target detection image belongs, including:

[0027] Obtaining a scale factor in a classifier of a data classification model, and performing a scale transformation on the fusion feature based on the scale factor; the scale factor is obtained by training the classifier in the data classification model;

[0028] Based on the classifier, key features corresponding to key features in the scale-transformed fusion features are identified, and based on the key identification results, the scale-transformed fusion features are classified to obtain the image category to which the target detection image belongs.

[0029] Among them, the fusion features are classified based on the key features to obtain the image category to which the target detection image belongs, including:

[0030] Classify the fused features based on the key features to obtain at least two prediction labels and a prediction probability value for each prediction label;

[0031] The predicted label with the largest predicted probability value is determined as the image category to which the target detection image belongs.

[0032] The method further comprises:

[0033] If the image category belongs to the image anomaly category, obtaining the communication method associated with the image category;

[0034] An object abnormality message is sent to the terminal device based on the communication method, so that the terminal device detects the object corresponding to the original image based on the object abnormality message; the object abnormality message includes the image category.

[0035] On the one hand, an embodiment of the present application provides a data classification method, the method comprising:

[0036] Obtain a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, and determine the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate; the sample visual identifier is used to indicate the visual angle of the detection image sample in the object corresponding to the detection image sample;

[0037] The initial convolutional neural network in the initial data classification model is used to obtain sample image features of the detection image samples, and the initial multi-layer perceptron in the initial data classification model is used to scale the detection sample position information to obtain sample position features of the detection image samples;

[0038] Perform feature splicing on sample image features and sample position features to obtain sample fusion features, determine sample key features of sample image features based on sample position features, classify sample fusion features based on sample key features, and obtain the image sample category to which the detected image sample belongs;

[0039] Based on the loss function between the image sample categories and the target sample labels, the initial data classification model is trained to obtain the data classification model.

[0040] Among them, the sample fusion features are classified based on the sample key features to obtain the image sample category to which the detected image sample belongs, including:

[0041] Classify and process the sample fusion features based on the sample key features to obtain at least two sample labels and a sample prediction probability value for each sample label; the at least two sample labels include a target sample label;

[0042] The sample label with the largest sample prediction probability value is determined as the image sample category to which the detected image sample belongs.

[0043] Wherein, the loss function includes a first loss function and a second loss function;

[0044] Based on the loss function between the image sample category and the target sample label, the initial data classification model is trained to obtain the data classification model, including:

[0045] Generate a first loss function according to the image sample category and the target sample label;

[0046] Generate a label distribution function according to the target sample label, generate a prediction distribution function according to at least two sample labels and the sample prediction probability value of each sample label, and generate a second loss function according to the label distribution function and the prediction distribution function;

[0047] The initial data classification model is trained according to the first loss function and the second loss function to obtain a data classification model.

[0048] In one aspect, an embodiment of the present application provides a data classification device, the device comprising:

[0049] An input acquisition module is used to acquire a visual identifier corresponding to an original image, identify a detection position coordinate used to extract an object of interest in the original image, acquire a target detection image containing the object of interest from the original image based on the detection position coordinate, and determine the detection position information corresponding to the original image based on the visual identifier and the detection position coordinate; the visual identifier is used to indicate a visual angle of the object of interest in the object corresponding to the original image;

[0050] A feature acquisition module is used to acquire image features of the target detection image, scale the detection position information, and obtain position features of the target detection image;

[0051] The data classification module is used to perform feature splicing on image features and position features to obtain fused features, determine key features in image features based on position features, classify fused features based on key features, and obtain the image category to which the target detection image belongs.

[0052] The input acquisition module includes:

[0053] A position determination unit, used to obtain an original image, perform object detection on the original image, obtain a first predicted frame containing the object of interest, and determine the detection position coordinates according to first position information of the first predicted frame;

[0054] The image determination unit is used to determine the second predicted border according to the detection position coordinates, and determine the area indicated by the second predicted border in the original image as the target detection image.

[0055] Wherein, in determining the detection position coordinates according to the first position information of the first predicted frame, the position determination unit includes:

[0056] A position acquisition subunit, used to determine a border width and a border height of the first predicted border according to the first position information of the first predicted border;

[0057] The size adjustment subunit is used to adjust the size of the first predicted border based on the border width and the border height, and determine the detection position coordinates according to the adjusted first predicted border.

[0058] The size adjustment subunit includes:

[0059] A position adjustment subunit, configured to adjust the size of the first predicted border according to the difference between the border width and the border height and the first position information, and determine the second position information and the adjusted width and height of the adjusted first predicted border;

[0060] The size transformation subunit is used to obtain the frame expansion coefficient, transform the adjustment width and the adjustment height based on the frame expansion coefficient, and determine the detection position coordinates according to the second position information and the transformed adjustment width and adjustment height.

[0061] Among them, in terms of acquiring image features of the target detection image, the feature acquisition module is specifically used for:

[0062] The target detection image is input into the convolutional neural network in the data classification model, and features of the target detection image are extracted based on the convolutional layer in the convolutional neural network to obtain image features of the target detection image.

[0063] Among them, in terms of performing scale transformation on the detection position information to obtain the position features of the target detection image, the feature acquisition module includes:

[0064] A position normalization unit, used to obtain the image width and image height of the target detection image, and perform normalization processing on the detection position coordinates based on the image width and image height to obtain normalized position coordinates;

[0065] A visual normalization unit, used for obtaining a visual total number, and performing normalization processing on the visual identification based on the visual total number to obtain a normalized visual identification;

[0066] The position feature acquisition unit is used to generate a perception input feature according to the normalized position coordinates and the normalized visual identifier, and use the multi-layer perceptron in the data classification model to scale the perception input feature to obtain the position feature of the target detection image.

[0067] Among them, in terms of performing feature splicing on image features and position features to obtain fusion features, the data classification module includes:

[0068] A feature normalization unit, used for normalizing the image features to obtain normalized image features, and normalizing the position features to obtain normalized position features;

[0069] The feature stitching unit is used to perform feature stitching on the normalized image features and the normalized position features to obtain fused features.

[0070] Among them, in terms of classifying the fused features based on the key features to obtain the image category to which the target detection image belongs, the data classification module includes:

[0071] A factor processing unit, used to obtain a scale factor in a classifier of a data classification model, and scale the fusion feature based on the scale factor; the scale factor is obtained by training the classifier in the data classification model;

[0072] The category determination unit is used to perform key recognition on features corresponding to key features in the scale-transformed fusion features based on a classifier, and classify the scale-transformed fusion features based on the key recognition results to obtain the image category to which the target detection image belongs.

[0073] Among them, in terms of classifying the fused features based on the key features to obtain the image category to which the target detection image belongs, the data classification module includes:

[0074] A result acquisition unit, used for classifying the fused features based on the key features to obtain at least two prediction labels and a prediction probability value for each prediction label;

[0075] The category determination unit is used to determine the prediction label with the largest prediction probability value as the image category to which the target detection image belongs.

[0076] The device further comprises:

[0077] A communication acquisition module, used for acquiring a communication method associated with an image category if the image category belongs to an image abnormality category;

[0078] The message sending module is used to send an object abnormality message to the terminal device based on the communication method, so that the terminal device detects the object corresponding to the original image based on the object abnormality message; the object abnormality message includes the image category.

[0079] In one aspect, an embodiment of the present application provides a data classification device, the device comprising:

[0080] A sample acquisition module is used to acquire a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, and determine the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate; the sample visual identifier is used to indicate the visual angle of the detection image sample in the object corresponding to the detection image sample;

[0081] A sample feature acquisition module is used to use the initial convolutional neural network in the initial data classification model to acquire sample image features of the detection image samples, and use the initial multi-layer perceptron in the initial data classification model to scale the detection sample position information to obtain sample position features of the detection image samples;

[0082] The sample classification module is used to perform feature splicing on the sample image features and the sample position features to obtain the sample fusion features, determine the sample key features of the sample image features based on the sample position features, and classify the sample fusion features based on the sample key features to obtain the image sample category to which the detection image sample belongs;

[0083] The model training module is used to train the initial data classification model based on the loss function between the image sample category and the target sample label to obtain the data classification model.

[0084] Among them, in terms of classifying the sample fusion features based on the sample key features to obtain the image sample category to which the detection image sample belongs, the sample classification module includes:

[0085] A probability acquisition unit, used to classify and process the sample fusion features based on the sample key features to obtain at least two sample labels and a sample prediction probability value of each sample label; the at least two sample labels include a target sample label;

[0086] The sample category determination unit is used to determine the sample label with the largest sample prediction probability value as the image sample category to which the detection image sample belongs.

[0087] Wherein, the loss function includes a first loss function and a second loss function;

[0088] The model training module includes:

[0089] A first loss generating unit, used to generate a first loss function according to the image sample category and the target sample label;

[0090] A second loss generating unit is used to generate a label distribution function according to the target sample label, generate a prediction distribution function according to at least two sample labels and the sample prediction probability value of each sample label, and generate a second loss function according to the label distribution function and the prediction distribution function;

[0091] The model generation unit is used to train the initial data classification model according to the first loss function and the second loss function to obtain the data classification model.

[0092] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input and output interface;

[0093] The processor is connected to the memory and the input and output interface respectively, wherein the input and output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer program to execute the data classification method in one aspect of an embodiment of the present application.

[0094] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the data classification method in one aspect of the embodiment of the present application.

[0095] In one aspect, an embodiment of the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional ways in one aspect of the embodiment of the present application.

[0096] Implementing the embodiments of the present application will have the following beneficial effects:

[0097] In an embodiment of the present application, a computer device obtains a visual identifier corresponding to an original image, identifies the detection position coordinates used to extract the object of interest in the original image, obtains a target detection image containing the object of interest from the original image based on the detection position coordinates, and determines the detection position information corresponding to the original image based on the visual identifier and the detection position coordinates; the visual identifier is used to represent the visual angle of the object of interest in the object corresponding to the original image; obtains image features of the target detection image, scales the detection position information, and obtains the position features of the target detection image; performs feature splicing on the image features and the position features to obtain fusion features, determines key features in the image features based on the position features, and classifies the fusion features based on the key features to obtain the image category to which the target detection image belongs. By combining the target detection image and the detection position information of the target detection image in the original image, it is possible to realize the recognition and classification of the target detection image based on the position information, so that under different position information, the recognition emphasis of the detection image is different, and the position corresponding to the acquired image category can be obtained, thereby improving the accuracy of data recognition and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0099] Figure 1 This is a network interaction architecture diagram of data classification provided by an embodiment of the present application;

[0100] Figure 2 This is a data classification model architecture diagram provided in an embodiment of the present application;

[0101] Figure 3 It is a flow chart of a method for data classification provided in an embodiment of the present application;

[0102] Figure 4a to Figure 4bThis is a schematic diagram of an object recognition scenario provided by an embodiment of the present application;

[0103] Figure 5 The embodiment of the present application provides a schematic diagram of a target detection image acquisition scenario;

[0104] Figure 6 This is a schematic diagram of an image recognition scenario provided by an embodiment of the present application;

[0105] Figure 7 This is a flow chart of a data classification model training method provided in an embodiment of the present application;

[0106] Figure 8 is a schematic diagram of a data classification device provided in an embodiment of the present application;

[0107] Fig. 9 is a schematic diagram of another data classification device provided in an embodiment of the present application;

[0108] Fig.10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0109] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0110] Optionally, the present application may adopt computer vision technology and deep learning technology in the field of artificial intelligence to realize the recognition of target detection images in original images and the recognition of image categories of target detection images.

[0111] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. For example, the acquisition of the target detection image in the original image and the detection position information of the target detection image, the recognition of the image category of the target detection image, etc. in this application can all be considered to be achieved by artificial intelligence.

[0112] Among them, artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technology and software-level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning and other major directions. And this application mainly involves computer vision technology (such as the recognition and acquisition of detection position information and target detection images in the original image) and machine learning / deep learning (such as the training and use of data classification models, etc.), among which, any one of the various technologies of artificial intelligence can be used alone in this application, and each technology in artificial intelligence can also be used in random combination, such as using computer vision technology alone, or combining computer vision technology with deep learning technology, etc., etc., which is not limited here. Through the use of related technologies of artificial intelligence, the efficiency of data recognition and classification in this application is improved.

[0113] Computer vision technology (CV) usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and common biometric recognition technologies such as face recognition and fingerprint recognition. Deep learning (DL) is a new research direction in the field of machine learning (ML). Deep learning is the inherent law and representation level of learning sample data. It is a complex machine learning algorithm. The results achieved in speech and image recognition far exceed previous related technologies. Deep learning usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0114] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role, such as the field of industrial defect detection in this application.

[0115] The solutions provided in the embodiments of the present application involve technologies such as computer vision and deep learning in the field of artificial intelligence, which are specifically described by the following embodiments:

[0116] In the examples of this application, see Figure 1 , Figure 1 It is a network interaction architecture diagram of data classification provided by an embodiment of the present application, and an embodiment of the present application can be implemented by a computer device. Among them, the computer device 101 can obtain the original image from the terminal device, identify the original image, and obtain the image category to which the target detection image in the original image belongs. Among them, the terminal device can be a terminal device 102a, a terminal device 102b or a terminal device 102c, and the computer device can exchange data with any terminal device, and each terminal device can exchange data with each other. Among them, the computer device 101 can obtain the original image from the terminal device, obtain the detection position information and the target detection image corresponding to the detection position information from the original image; or, the terminal device obtains the detection position information and the target detection image corresponding to the detection position information from the original image, and sends the detection position information and the target detection image to the computer device; or, the computer device can directly obtain the original image, obtain the detection position information and the target detection image corresponding to the detection position information from the original image, etc., without limitation.

[0117] For details, see Figure 2 , Figure 2 This is a data classification model architecture diagram provided by the embodiment of the present application. Figure 2 As shown, the computer device obtains the detection position coordinates for extracting the object of interest in the original image, obtains the visual identifier corresponding to the original image, and generates the detection position information 2012 according to the visual identifier and the detection position coordinates. The computer device can obtain the target detection image 2011 containing the object of interest from the original image based on the detection position coordinates, and input the target detection image 2011 and the detection position information 2012 into the data classification model 202, wherein the data classification model 202 includes a convolutional neural network 2021 and a multi-layer perceptron 2022. Specifically, the target detection image 2011 is input into the convolutional neural network 2021, and the image features of the target detection image 2011 are obtained based on the convolutional neural network 2021; the detection position information 2012 is input into the multi-layer perceptron 2022, and the detection position information is scaled based on the multi-layer perceptron 2022 to obtain the position features of the target detection image 2011. The computer device performs feature splicing on the image features and the position features to obtain fused features, and can determine the key features in the image features based on the position features, and classify the fused features based on the key features to obtain the image category to which the target detection image 2011 belongs. By combining the detection position information and the target detection image, the target detection image can be identified and detected based on the emphasis of the detection position information (i.e., the key features), and the corresponding position of the image category in the original image is used, thereby improving the accuracy of data classification and recognition.

[0118] It is understandable that the computer devices mentioned in the embodiments of the present application include but are not limited to terminal devices or servers. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, PDAs, augmented reality / virtual reality (AR / VR) devices, helmet displays, wearable devices, smart speakers, digital cameras, cameras and other mobile Internet devices (mobile internet device, MID) with network access capabilities, etc., wherein the client has a display function. Among them, the server mentioned above can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0119] Optionally, the data involved in the embodiments of the present application can be stored in a computer device, or the data can be stored based on cloud storage technology, which is not limited here. Among them, the computer device can cache the original image or detection position information and target detection image after acquiring the original image or detection position information and target detection image, and process the cached original image or detection position information and target detection image based on the data classification cycle; it can also process the acquired original image or detection position information and target detection image in real time, which is not limited here.

[0120] For further information, see Figure 3 , Figure 3 This is a flow chart of a method for data classification provided by an embodiment of the present application. Figure 3 As shown, an original image is used as an example for description. In other words, Figure 3 In the described method embodiment, the data classification process includes the following steps:

[0121] Step S301, obtain the visual identifier corresponding to the original image, identify the detection position coordinates in the original image for extracting the object of interest, obtain a target detection image containing the object of interest from the original image based on the detection position information, and determine the detection position information corresponding to the original image based on the visual identifier and the detection position coordinates.

[0122] In an embodiment of the present application, a computer device may obtain an original image, obtain detection position coordinates for extracting an object of interest from the original image, and obtain a target detection image containing the object of interest from the original image based on the detection position coordinates, wherein the object of interest refers to the object to be identified in the present application. The computer device may also obtain a visual identifier corresponding to the original image, and determine the detection position information corresponding to the original image according to the visual identifier and the detection position coordinates. For example, in the field of industrial defect detection, the object of interest may be an industrial defect; in the field of physical object recognition, the object of interest may be a person, an animal, or an object, etc.; in other words, in different fields, the corresponding objects of interest are different, and the object of interest may be changed as needed. Among them, the visual identifier is used to indicate the visual angle of the object of interest in the object corresponding to the original image. For example, the object corresponding to the original image has four visual angles, corresponding to visual angle 1, visual angle 2, visual angle 3, and visual angle 4, respectively. When the visual angle 2 corresponding to the object is photographed and the original image is generated, the visual identifier corresponding to the original image is the identifier of the visual angle 2, and the object of interest is the form presented by the object at the visual angle 2.

[0123] Furthermore, the computer device can obtain the original image, perform object detection on the original image, obtain a first predicted bounding box including the object of interest, determine the detection position coordinates based on the first position information of the first predicted bounding box; determine the second predicted bounding box based on the detection position coordinates, and determine the area indicated by the second predicted bounding box in the original image as the target detection image.

[0124] The process of determining the first predicted border can be found in Figure 4a to Figure 4b , Figure 4a to Figure 4b is a schematic diagram of an object recognition scenario provided by an embodiment of the present application, wherein: Figure 4a is a schematic diagram of a scene for determining an object recognition point provided in an embodiment of the present application. Figure 4b Schematic diagram of a frame determination scenario provided by an embodiment of the present application. Figure 4a As shown, the computer device can obtain the original image 401, divide the original image 401 into a number of equal recognition frames, that is, obtain at least two recognition frames, obtain the correlation between each recognition frame and the object of interest, and determine the recognition point based on the correlation. Assuming that three possible objects of interest are identified from the original image, they are recorded as object of interest 1, object of interest 2, and object of interest 3, wherein the recognition point corresponding to object of interest 1 is determined as recognition point 4021, the recognition point corresponding to object of interest 2 is determined as recognition point 4022, and the recognition point corresponding to object of interest 3 is determined as recognition point 4023. Further, as Figure 4bAs shown, the identification point 4021 is transformed in size and shape, etc., to obtain at least two associated frames for the possible object of attention 1, including but not limited to associated frame 4031, associated frame 4032 and associated frame 4033; the identification point 4022 is transformed in size and shape, etc., to obtain at least two associated frames for the possible object of attention 2, including but not limited to associated frame 4041 and associated frame 4042; the identification point 4023 is transformed in size and shape, etc., to obtain at least two associated frames for the possible object of attention 3, including but not limited to associated frame 4051 and associated frame 4052. Obtain the confidences corresponding to the associated borders 4031, 4032 and 4033 respectively. Assuming that the confidence of the associated border 4032> the confidence of the associated border 4031> the confidence of the associated border 4033, determine the associated border 4032 as the first predicted border corresponding to the focus object 1; obtain the confidences corresponding to the associated borders 4041 and 4042 respectively. Assuming that the confidence of the associated border 4042> the confidence of the associated border 4041, determine the associated border 4042 as the first predicted border corresponding to the focus object 2; obtain the confidences corresponding to the associated borders 4051 and 4052 respectively. Assuming that the confidence of the associated border 4052> the confidence of the associated border 4051, determine the associated border 4052 as the first predicted border corresponding to the focus object 3. According to the first predicted bounding box of the object of interest 1, the first position coordinates of the object of interest 1 are determined, and the detection position coordinates of the object of interest 1 are determined according to the first position information. Similarly, the detection position coordinates of the object of interest 2 and the detection position coordinates of the object of interest 3 are obtained. Among them, the confidence is used to indicate the probability of the object of interest appearing in the corresponding associated bounding box.

[0125] Optionally, the computer device may obtain the original image, generate at least two pre-selected boxes based on the original image, wherein overlapping portions may appear between at least two pre-selected boxes, obtain the confidence of each pre-selected box, and determine the pre-selected box with a confidence greater than a preset confidence threshold as the first predicted bounding box. For example, in the field of industrial defect detection, the object of interest is an industrial defect, and the computer device may obtain the confidence of each pre-selected box, which is used to indicate the possibility of the existence of an industrial defect in the corresponding pre-selected box, and determine the first predicted bounding box where the industrial defect may exist from the at least two pre-selected boxes based on the confidence of each pre-selected box.

[0126] Optionally, the computer device may also identify the original image based on the target detection model, identify the detection position coordinates for extracting the object of interest in the original image, and obtain the target detection image containing the object of interest from the original image based on the detection position coordinates. The target detection model may be a fast region convolutional neural network (Faster R-CNN), a single shot multi-box detector (SingleShot MultiBox Detector, SSD) or a single detection model (You Only Look Once, YOLO), etc., without limitation here.

[0127] Furthermore, after acquiring the first predicted border in the original image, the computer device can determine the detection position coordinates according to the first position information of the first predicted border. Specifically, the computer device can determine the border width and border height of the first predicted border according to the first position information of the first predicted border; based on the border width and border height, the first predicted border is resized, and the detection position information is determined according to the adjusted first predicted border. Among them, the area indicated by the detection position coordinates in the original image can be called the Region of Interest (ROI), that is, the area that needs to be identified in this application. In the region of interest, a sufficiently appropriate background is generally required to contrast the object of interest. Therefore, the first predicted border can be resized so that the area indicated by the adjusted first predicted border can include the object of interest and the background used to contrast the object of interest, thereby improving the accuracy of data recognition and classification. Optionally, a fully connected layer is used in the data classification model to obtain the image category to which the target detection image belongs. Therefore, the size of the target detection image obtained can be made consistent with the size that the data classification model can recognize by resizing and reshaping the first predicted bounding box, so as to ensure that no distortion occurs when the target detection image is scaled in the data classification model, thereby improving the accuracy of data recognition and classification. Based on this, when the computer device resizes the first predicted bounding box, it can first convert the first predicted bounding box into a square, and then scale the square to obtain the detection position coordinates; or, the computer device can convert the first predicted bounding box into a square, and obtain the detection position coordinates based on the square; or, the computer device can scale the first predicted bounding box to obtain the detection position coordinates.

[0128] Among them, taking the example of a computer device first converting the first predicted border into a square and then scaling the square to obtain the detection position coordinates, the computer device can adjust the size of the first predicted border according to the difference between the border width and the border height and the first position information, and determine the second position information and the adjusted width and height of the adjusted first predicted border; obtain the border expansion coefficient, transform the adjustment width and height based on the border expansion coefficient, and determine the detection position coordinates according to the second position information and the transformed adjustment width and height.

[0129] Among them, the first position information of the first predicted frame is recorded as Represents the position coordinates of the upper left vertex of the first predicted bounding box in the original image, Indicates the position coordinates of the lower right vertex of the first predicted bounding box in the original image. For details, see Figure 5 , Figure 5 Schematic diagram of a target detection image acquisition scenario provided by an embodiment of the present application. Figure 5 As shown, after the computer device performs object detection on the original image 501, it obtains a first predicted border 5011 containing the object of interest, and the four vertices of the first predicted border 5011 are respectively denoted as a d , b d 、c d and d , where the upper left vertex a d The position coordinates in the original image 501 are Lower right vertex c d The position coordinates in the original image 501 are According to the first position information of the first predicted border 5011, the border width w of the first predicted border 5011 is determined d And the frame height h d , where the border width w d The calculation formula of can be seen in formula (1), the border height h d See formula (2):

[0130]

[0131]

[0132] Among them, if the border height h d Greater than the border width w d , then for the border width w d Expand; if the border width w d Greater than the border height h d , then for the border height h dExpand, where the number of expanded pixels is abs refers to the operation of taking absolute value. Figure 5 As shown in d Greater than the border width w d , then the border width w d Expand p pixels to both sides respectively to obtain the second position information and the adjusted width and height of the adjusted first prediction frame 502. The second position information is recorded as (x′ min ,y′ min ,x′ max ,y′ max ), then the method for obtaining the second position information can refer to formula (3) to formula (6):

[0133]

[0134]

[0135]

[0136]

[0137] According to the second position information, the adjusted width and height of the adjusted first prediction frame 502 are determined, the adjusted width is recorded as w', the adjusted height is recorded as h', the four vertices of the adjusted first prediction frame 502 are recorded as a', b', c' and d' respectively, and the position coordinates of the upper left vertex of the adjusted first prediction frame 502 are (x' min ,y′ min ), the position coordinate of the lower right vertex of the adjusted first prediction frame 502 is (x′ max ,y′ max ). The method for obtaining the adjusted width w' can be referred to as shown in formula (7), and the method for obtaining the adjusted height h' can be referred to as shown in formula (8):

[0138] w′=x′ max -x′ min +1 (7)

[0139] h′=y′ max -y′ min +1 (8)

[0140] Furthermore, the computer device can obtain a border expansion coefficient, and resize the adjusted width w' and the adjusted height h' based on the border expansion coefficient. The border expansion coefficient is recorded as r. The border expansion coefficient r is a trainable parameter determined in the process of training the data classification model. After testing, when the border expansion coefficient r is 0.4, the performance of the data classification model is optimal. Of course, with the continuous optimization of the data classification model, the value of the border expansion coefficient r can also be updated so that the data classification model can obtain better performance. Among them, the detection position information is determined according to the second position information and the transformed adjusted width w' and adjusted height h', and the detection position information is recorded as (x min ,y min ,x max ,y max ), the method for obtaining the detection position information can be referred to as shown in formula (9) to formula (12):

[0141] x min = x′ min -w′*r (9)

[0142] y min =y′ min -h′*r (10)

[0143] x max = x′ max +w′*r (11)

[0144] y max =y′ max +h′*r (12)

[0145] The second prediction frame 503 is determined according to the detection position information. The four vertices of the second prediction frame 503 are a, b, c and d respectively. The position coordinate of the upper left vertex a of the second prediction frame 503 is (x min ,y min ), the position coordinate of the lower right vertex c of the second prediction frame 503 is (x max ,y max ), the width of the second prediction frame 503 can be recorded as w, and the height can be recorded as h. The area indicated by the second prediction frame 503 in the original image 501 is determined as the target detection image 504, and the image width of the target detection image 504 is w, and the image height is h.

[0146] Optionally, if the second predicted border 503 is in the original image 501 and exceeds the image boundary of the original image 501, assuming that the original image 501 takes the upper left vertex as the coordinate origin, the width of the original image 501 is w p , height h p , such as when xmin <0, or, y min <0, or, x max >w p , or, max >h p When the second prediction border 503 is determined to exceed the image boundary of the original image 501, the area indicated by the second prediction border 503 that exceeds the image boundary of the original image 501 is recorded as an out-of-bounds area, and a default value is added to the out-of-bounds area to obtain a target detection image.

[0147] Optionally, the computer device may generate the detection position information according to the visual identifier and the detection position coordinates. The computer device may determine the detection position coordinates according to the second position information and the transformed adjusted width and adjusted height. In this case, the detection position coordinates are (x min ,y min ,x max ,y max ), the computer device can obtain the visual identifier, and determine the detection position information according to the visual identifier and the detection position coordinates. The visual identifier can be the visual identifier corresponding to the original image. For example, all cameras involved in the shooting are numbered, such as N cameras are numbered 1, 2, 3, ... and N respectively, and the number of the camera associated with the visual image corresponding to the original image can be determined as the visual identifier of the original image. For example, if the original image is taken by camera 2, the visual identifier of the original image can be determined to be 2. According to the visual identifier and the detection position coordinates, the detection position information is determined, and the detection position information is (2, x min ,y min ,x max ,y max ). Where N is a positive integer. Optionally, the visual identifier can also be determined based on the visual position. For example, an object is photographed from N visual positions to obtain N original images. The N original images include original image i. The visual identifier of original image i is the identifier corresponding to visual position i. i is a positive integer and i is less than or equal to N. The above are two optional ways to determine the visual identifier. Other identifiers that can represent the shooting vision of the original image can also be used as the visual identifier of the original image, which is not limited here. There may be multiple original images. Figure 3 In the illustrated embodiment, an original image is taken as an example for description.

[0148] Step S302, acquiring image features of the target detection image, performing scale transformation on the detection position information, and obtaining position features of the target detection image.

[0149] In an embodiment of the present application, the computer device can input the target detection image into the convolutional neural network (CNN) in the data classification model, and extract the features of the target detection image based on the convolutional layer in the convolutional neural network to obtain the image features of the target detection image. The CNN can be any neural network that can obtain image features, such as a ResNet18 network, etc., without limitation, wherein the numbers in the ResNet18 network represent the depth of the network, referring to weighted layers, including convolutional layers and fully connected layers, excluding pooling layers and batch normalization (BatchNormalization, BN) layers, etc.

[0150] The detection position information may include a visual identifier and a detection position coordinate. The detection position information is scaled to obtain the position feature of the target detection image, including: obtaining the image width and image height of the target detection image, normalizing the detection position coordinate based on the image width and image height to obtain the normalized position coordinate; obtaining the total number of visuals, normalizing the visual identifier based on the total number of visuals to obtain the normalized visual identifier; generating a perceptual input feature according to the normalized position coordinate and the normalized visual identifier, and using a multi-layer perceptron in the data classification model to scale the perceptual input feature to obtain the position feature of the target detection image. The scale refers to a criterion, which can represent a standard for looking at things. Since the visual identifier and the detection position coordinate have different scales, that is, the visual identifier and the detection position coordinate have different measurement standards, if the two are directly combined and input into the data classification model for prediction, it will make the data classification model difficult to predict and may also cause inaccurate prediction. Therefore, the visual identifier and the detection position coordinate can be normalized so that the visual identifier and the detection position coordinate have the same scale, thereby improving the prediction accuracy and prediction efficiency of the data classification model. The normalization process of the visual logo can be shown in formula (13):

[0151]

[0152] Among them, the ID cam Refers to the visual identification of the original image. Here, the visual identification is taken as the camera number corresponding to the original image, that is, ID cam It can take any value from 1 to N, where N is the total number of visuals. The visual identity is normalized based on the total number of visuals to obtain the normalized visual identity.

[0153] Furthermore, the normalization process of the detection position coordinates can be shown in formulas (14) to (17):

[0154]

[0155]

[0156]

[0157]

[0158] Among them, w is the image width of the target detection image, h is the image height of the target detection image, (x min ,y min ,x max ,y max ) is the detection position coordinate, and the detection position coordinate is normalized based on the image width and image height to obtain the normalized position coordinate

[0159] Generate perceptual input features based on normalized position coordinates and normalized visual identifiers The multi-layer perceptron (MLP) in the data classification model is used to scale the perceptual input features to obtain the position features of the target detection image.

[0160] Among them, it is assumed that the image feature is a 512-dimensional feature, and the detection position information is a 5-dimensional feature (i.e., the visual identifier and the detection position coordinate). There is a large scale difference between the two, and the dimensional difference is large, which causes the position feature to be almost negligible when predicting in the data classification model. The position feature is obtained by scaling the perceptual input feature through MLP, which reduces the dimensional difference between the image feature and the position feature, thereby improving the dimensional adaptation problem between the two and further improving the prediction accuracy of the data classification model. Furthermore, the computer device can obtain the image dimension information of the image feature, determine the position dimension range according to the image dimension information, use MLP to scale the perceptual input feature, and obtain the position feature of the target detection image based on the position dimension range, and the position dimension information of the position feature belongs to the position dimension range. For example, the position dimension range is greater than the first proportion (such as 1 / 4) of the image dimension information and less than the image dimension information, that is, the ratio of the position dimension information of the position feature to the image dimension information is greater than the first proportion (1 / 4) and less than 1. This is because when the target detection image is identified and classified, the importance of the features of the image itself is higher. While ensuring the dominant position of the image features in the prediction process, the scale adaptability between the position features and the image features is improved, and the scale transformation of the perceived input features to a certain extent can be performed based on the position dimension range. Optionally, the position dimension range can also be updated as needed. For example, in the process of predicting the target detection image, when the importance of the position feature is higher than that of the image feature, the position dimension range can be greater than the image dimension information and less than the second proportion (such as 5 / 4) of the image dimension information, or the position dimension range can be greater than the first proportion of the image dimension information and less than the second proportion of the image dimension information, which is not limited here. For example, in the field of industrial defect detection, in order to ensure that industrial defects can be detected, it can be considered that the importance of the image feature is greater than the position feature, and the position dimension range can be greater than the first proportion of the image dimension information and less than the image dimension information.

[0161] Step S303, perform feature splicing on the image features and the position features to obtain fused features, determine key features in the image features based on the position features, classify the fused features based on the key features, and obtain the image category to which the target detection image belongs.

[0162] In the embodiment of the present application, the computer device can normalize the image features to obtain normalized image features, normalize the position features to obtain normalized position features, and perform feature splicing on the normalized image features and the normalized position features to obtain fusion features. By normalizing the image features and the position features, the scale difference between the image features and the position features is further reduced, and the prediction accuracy of the data classification model is improved. The process of normalizing the image features can be shown in formula (18):

[0163]

[0164] Among them, F CNN represents image features, normalize is the normalization algorithm, and the position feature is recorded as F MLP , the process of normalizing the position features can be seen in formula (19):

[0165]

[0166] Further, the computer device may perform feature splicing on the normalized image features and the normalized position features to obtain a fused feature. Optionally, the computer device may also perform feature splicing on the normalized image features and the normalized position features, and perform normalization processing on the splicing result to obtain a fused feature. In this case, the method for obtaining the fused feature may refer to formula (20):

[0167]

[0168] Wherein, concat represents a feature concatenation operation, and the normalization algorithm can be an L1 normalization algorithm, an L2 normalization algorithm, or a linear function normalization algorithm, etc., which are not limited here. The input feature of the normalize is defined as X = (x1, x2, ..., x m ), taking the L2 normalization algorithm as an example, the normalization algorithm can be shown in formula (21):

[0169]

[0170] Wherein, m is a positive integer, and m can represent the dimension of the input feature. For example, in formula (18), m represents the dimension of the image feature, in formula (19), m represents the dimension of the position feature, and in formula (20), m can represent the dimension of the fusion feature.

[0171] Furthermore, the computer device can obtain the scale factor in the classifier of the data classification model, and scale the fused features based on the scale factor, wherein the scale factor is obtained by training the classifier in the data classification model; based on the classifier, the features corresponding to the key features in the scale-transformed fused features are key-identified, and the scale-transformed fused features are classified based on the key identification results to obtain the image category to which the target detection image belongs. In other words, the key identification refers to focusing on the features corresponding to the key features, for example, increasing the weight of the features corresponding to the key features. Among them, the modulus length of the normalized features is 1. After the fused features are input into the classifier, if the activation function is directly used to predict the fused features, the output value may be too small, resulting in difficulty in model training and the model not converging. Therefore, the scale factor s can be added to the classifier of the data classification model, and then the fused features can be predicted based on the classifier, thereby improving the accuracy of data recognition and classification. Among them, if the value of the scale factor s is too small, the scale transformation of the fusion feature based on the scale factor s is not helpful for the training of the data classification model; if the value of the scale factor s is too large, the predicted probability value of the output prediction label is likely to be too large, or even close to 1, resulting in the problem of overconfidence in the classification result. Therefore, the scale factor s can be defined as a learnable parameter rather than a manually adjusted hyperparameter, so that the scale factor s can optimize the data classification model and improve the accuracy of the data classification model in identifying and classifying data. Among them, in the training process of the data classification model, the scale factor s is learned. After experiments, when the value of the scale factor s is between 2 and 5, the effect of the data classification model is better. Optionally, when the data classification model continues to be learned and optimized, the value range of the scale factor s can be optimized and updated.

[0172] Among them, when the computer device classifies the fused features to obtain the image category to which the target detection image belongs, it may not perform scale transformation on the fused features, but directly classify the fused features based on the key features to obtain at least two prediction labels and the prediction probability value of each prediction label; the prediction label with the largest prediction probability value is determined as the image category to which the target detection image belongs. Alternatively, the computer device may classify the scale-transformed fused features based on the classifier to obtain at least two prediction labels and the prediction probability value of each prediction label, and determine the prediction label with the largest prediction probability value as the image category to which the target detection image belongs.

[0173] Furthermore, if the image category belongs to the image abnormality category, the communication method associated with the image category is obtained; an item abnormality message is sent to the terminal device based on the communication method, so that the terminal device detects the item corresponding to the original image based on the item abnormality message; the item abnormality message includes the image category. For example, see Figure 6 , Figure 6 Schematic diagram of an image recognition scenario provided by an embodiment of the present application. Figure 6 As shown, the computer device inputs the target detection image 601 into the convolutional neural network of the data classification model 602, extracts features of the target detection image 601 based on the convolutional neural network, and obtains the image features of the target detection image 601; inputs the detection position information into the multi-layer perceptron of the data classification model 602, and performs scale transformation on the detection position information based on the multi-layer perceptron to obtain the position features of the target detection image. The computer device performs feature splicing on the image features and the position features to obtain fused features 6021, and uses the classifier in the data classification model 602 to classify the fused features 6021 to obtain at least two predicted labels 6022 and the predicted probability value of each predicted label, and determines the predicted label with the maximum predicted probability value as the image category 603 to which the target detection image belongs. It is assumed that the at least two prediction labels 6022 include (K+1) prediction labels, which are respectively recorded as (prediction label 1, prediction label 2, ..., prediction label K and prediction label (K+1)), wherein the at least two prediction labels 6022 include regular labels and K types of image abnormality labels. If the image category 603 belongs to the K types of image abnormality labels, it means that the image category 603 belongs to the image abnormality category, and an item abnormality message is sent to the terminal device 604. Taking the field of industrial defect detection as an example, assuming that the K types of image abnormality labels are industrial cracks, inclusions, pressed oxide scale, pits, plaques and oil interference, etc., and assuming that the image category 603 is the image abnormality label corresponding to "industrial cracks", then the communication method for processing the "industrial cracks" is obtained, and based on the communication method, an item abnormality message is sent to the terminal device indicated by the communication method, so that the terminal device detects the item corresponding to the original image based on the item abnormality message. If the terminal device is an automatic processing device, the terminal device detects the item corresponding to the original image and repairs the "industrial cracks"; or, the communication method is associated with a staff member, and after the staff member obtains the item abnormality message from the terminal device, they detect and repair the item corresponding to the original image.

[0174] In an embodiment of the present application, a computer device obtains a visual identifier corresponding to an original image, identifies the detection position coordinates used to extract the object of interest in the original image, obtains a target detection image containing the object of interest from the original image based on the detection position coordinates, and determines the detection position information corresponding to the original image based on the visual identifier and the detection position coordinates; the visual identifier is used to represent the visual angle of the object of interest in the object corresponding to the original image; obtains image features of the target detection image, scales the detection position information, and obtains the position features of the target detection image; performs feature splicing on the image features and the position features to obtain fusion features, determines key features in the image features based on the position features, and classifies the fusion features based on the key features to obtain the image category to which the target detection image belongs. By combining the target detection image and the detection position information of the target detection image in the original image, it is possible to realize the recognition and classification of the target detection image based on the position information, so that under different position information, the recognition emphasis of the detection image is different, and the position corresponding to the acquired image category can be obtained, thereby improving the accuracy of data recognition and classification.

[0175] For further information, see Figure 7 , Figure 7 Schematic diagram of a data classification model training method provided in an embodiment of the present application. Figure 7 As shown, the method comprises the following steps:

[0176] Step S701, obtaining a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, and determining the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate.

[0177] In an embodiment of the present application, a computer device obtains a training sample, which may include a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, and determines the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate, wherein the target sample label is used to indicate the sample label actually corresponding to the detection image sample. Optionally, the computer device may perform data splicing of the sample visual identifier and the detection sample position coordinate to obtain the detection sample position information.

[0178] Step S702, using the initial convolutional neural network in the initial data classification model to obtain sample image features of the detection image samples, using the initial multi-layer perceptron in the initial data classification model to scale the detection sample position information to obtain sample position features of the detection image samples.

[0179] In the embodiment of the present application, the process of obtaining the sample image features can be referred to in Figure 3The process of obtaining the image features in step S302 and the process of obtaining the sample position features can be found in Figure 3 The process of obtaining the position feature in step S302 will not be described in detail here.

[0180] Step S703, perform feature splicing on the sample image features and the sample position features to obtain sample fusion features, determine the sample key features of the sample image features based on the sample position features, classify the sample fusion features based on the sample key features, and obtain the image sample category to which the detected image sample belongs.

[0181] In the embodiment of the present application, the computer device can classify the sample fusion features based on the sample key features to obtain at least two sample labels and a sample prediction probability value for each sample label; the at least two sample labels include a target sample label; and the sample label with the largest sample prediction probability value is determined as the image sample category to which the detected image sample belongs. The process can be specifically referred to in Figure 3 As shown in step S303.

[0182] Step S704: training the initial data classification model based on the loss function between the image sample categories and the target sample labels to obtain a data classification model.

[0183] In an embodiment of the present application, the loss function includes a first loss function and a second loss function. The computer device can generate a first loss function according to the image sample category and the target sample label; generate a label distribution function according to the target sample label, generate a prediction distribution function according to at least two sample labels and the sample prediction probability value of each sample label, and generate a second loss function according to the label distribution function and the prediction distribution function; train the initial data classification model according to the first loss function and the second loss function to obtain a data classification model. The first loss function can be shown in formula (22):

[0184] L1=CrossEntroyLoss(logits) (22)

[0185] Wherein, logits is the output of the classifier in the data classification model. For example, if there are C sample labels that need to be classified, then logits is a vector containing C elements. CrossEntroyLoss is a first loss function used to perform classification learning on the data classification model. The first loss function can be a cross entropy loss function, etc., which is not limited here. The second loss function can be shown in formula (23):

[0186] L2=KLDivLoss(logits) (23)

[0187] The KLDivLoss is a second loss function used for distribution learning of the data classification model. The second loss function may be a KL divergence (Kullback-Leibler divergence) loss function, which is used to calculate the degree of difference between two random variables. The loss function may be shown in formula (24):

[0188] L=L1+λ*L2 (24)

[0189] Among them, λ is the weight coefficient between the first loss function and the second loss function. The weight coefficient can be an empirical value or can be optimized and determined during the training process of the data classification model.

[0190] Furthermore, the process of training the data classification model is as follows:

[0191] In the model deployment stage of the data classification model, the loss function layer is first removed, and steps S701 to S703 are executed. The logits are converted into a probability distribution through the initial classifier, that is, at least two sample labels and the sample prediction probability value of each sample label are obtained. The at least two sample labels and the sample prediction probability value of each sample label can be represented by a vector probs. Among them, the classifier can convert logits into a probability distribution through an activation function, and the activation function can be a logistic regression (softmax) function. The process is shown in formula (25):

[0192] probs = softmax(logits) (25)

[0193] The sample label with the largest sample prediction probability value is determined as the image sample category to which the detected image sample belongs. The process is shown in formula (26):

[0194] label = argmax(probs) (26)

[0195] Among them, argmax represents the index corresponding to the largest element in the vector, that is, the sample label corresponding to the largest sample prediction probability value, which is determined as the image sample category label to which the detected image sample belongs. The sample prediction probability value corresponding to the image sample category label is the confidence of the image sample category, which can be expressed as formula (27):

[0196] score = probs[label] (27)

[0197] Among them, probs[label] represents the value corresponding to the image sample category label in the vector probs.

[0198] Furthermore, the computer device can set an initial distribution function. If the distribution of at least two sample labels is ordered (such as age prediction of a face, etc.), the initial distribution function can be a conventional probability distribution function, such as a Gaussian function, etc. The image sample category is substituted into the initial distribution function to generate a sample distribution function; the probability mean and probability variance of at least two sample labels and the sample prediction probability value of each sample label are obtained, and the training distribution function is determined based on the probability mean and probability variance. If the distribution of the at least two sample labels is disordered, the number of each sample label in the training set as the image sample category can be counted, and the training proportion of each sample label in the training set can be determined according to the number of each sample label as the image sample category, and the training proportion of each sample label in the training set can be used as the training distribution function; for example, assuming that there are 4 sample labels, in the training set, sample label 1 is used as the image sample category 10 times, sample label 2 is used as the image sample category 30 times, sample label 3 is used as the image sample category 35 times, and sample label 4 is used as the image sample category 25 times, then the training distribution function can be determined to be (0.1, 0.3, 0.35, 0.25). The first loss function is used to determine the first loss value between the image sample category and the target sample label, and the second loss function is used to determine the second loss value between the sample distribution function and the training distribution function. The total loss value between the first loss function and the second loss function is determined by formula (24), and the initial data classification model is trained based on the total loss value to obtain the data classification model.

[0199] In an embodiment of the present application, a computer device can train a data classification model in combination with classification learning and distribution learning. Classification learning can truly reflect the predicted category confidence of the data classification model, and distribution learning can ensure that the model can be trained smoothly, thereby improving the training reliability of the data classification model.

[0200] Among them, referring to Table 1, Table 1 shows the performance of a certain defect classification. The first performance is the performance when visual identification and ROI detection position coordinates are not used, and the second performance is the performance when visual identification and detection position coordinates are integrated in this application. The larger the indicator value, the better the performance.

[0201] Table 1

[0202]

[0203] It can be seen that, based on the existing technology, the performance of this application has been improved to a certain extent, and the accuracy of data recognition and classification can be improved.

[0204] The training process and prediction process of the data classification model can be executed by the same computer device or by different computer devices.

[0205] For further information, see Figure 8 , Figure 8 Schematic diagram of a data classification device provided in an embodiment of the present application. The data classification device may be a computer program (including program code, etc.) running in a computer device, for example, the data classification device may be an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Figure 8 As shown, the data classification device 800 can be used for Figure 3 The computer device in the corresponding embodiment may specifically include: an input acquisition module 11 , a feature acquisition module 12 and a data classification module 13 .

[0206] The input acquisition module 11 is used to acquire the visual identifier corresponding to the original image, identify the detection position coordinates used to extract the object of interest in the original image, acquire the target detection image containing the object of interest from the original image based on the detection position coordinates, and determine the detection position information corresponding to the original image according to the visual identifier and the detection position coordinates; the visual identifier is used to indicate the visual angle of the object of interest in the object corresponding to the original image;

[0207] A feature acquisition module 12 is used to acquire image features of the target detection image, scale the detection position information, and obtain position features of the target detection image;

[0208] The data classification module 13 is used to perform feature splicing on the image features and the position features to obtain fused features, determine the key features in the image features based on the position features, classify the fused features based on the key features, and obtain the image category to which the target detection image belongs.

[0209] The input acquisition module 11 includes:

[0210] The position determination unit 111 is used to obtain an original image, perform object detection on the original image, obtain a first predicted bounding box containing the object of interest, and determine the detection position coordinates according to first position information of the first predicted bounding box;

[0211] The image determination unit 112 is used to determine a second predicted bounding box according to the detection position coordinates, and determine the area indicated by the second predicted bounding box in the original image as the target detection image.

[0212] In terms of determining the detection position coordinates according to the first position information of the first predicted frame, the position determination unit 111 includes:

[0213] The position acquisition subunit 1111 is used to determine the frame width and frame height of the first predicted frame according to the first position information of the first predicted frame;

[0214] The size adjustment subunit 1112 is used to adjust the size of the first predicted border based on the border width and the border height, and determine the detection position coordinates according to the adjusted first predicted border.

[0215] The size adjustment subunit 1112 includes:

[0216] The position adjustment subunit 111a is used to adjust the size of the first predicted border according to the difference between the border width and the border height and the first position information, and determine the second position information and the adjusted width and height of the adjusted first predicted border;

[0217] The size conversion subunit 111b is used to obtain a frame expansion coefficient, perform size conversion on the adjustment width and the adjustment height based on the frame expansion coefficient, and determine the detection position coordinates according to the second position information and the converted adjustment width and adjustment height.

[0218] In terms of acquiring image features of the target detection image, the feature acquisition module 12 is specifically used for:

[0219] The target detection image is input into the convolutional neural network in the data classification model, and features of the target detection image are extracted based on the convolutional layer in the convolutional neural network to obtain image features of the target detection image.

[0220] In terms of performing scale transformation on the detection position information to obtain the position features of the target detection image, the feature acquisition module 12 includes:

[0221] The position normalization unit 121 is used to obtain the image width and image height of the target detection image, and perform normalization processing on the detection position coordinates based on the image width and image height to obtain normalized position coordinates;

[0222] A visual normalization unit 122, used to obtain a visual total number, and normalize the visual identifier based on the visual total number to obtain a normalized visual identifier;

[0223] The position feature acquisition unit 123 is used to generate a perception input feature according to the normalized position coordinates and the normalized visual identifier, and use a multi-layer perceptron in the data classification model to scale the perception input feature to obtain the position feature of the target detection image.

[0224] Among them, in terms of performing feature splicing on the image features and the position features to obtain fusion features, the data classification module 13 includes:

[0225] A feature normalization unit 131 is used to perform normalization processing on the image features to obtain normalized image features, and to perform normalization processing on the position features to obtain normalized position features;

[0226] The feature stitching unit 132 is used to perform feature stitching on the normalized image features and the normalized position features to obtain fused features.

[0227] In terms of classifying the fused features based on the key features to obtain the image category to which the target detection image belongs, the data classification module 13 includes:

[0228] A factor processing unit 133 is used to obtain a scale factor in a classifier of a data classification model, and scale the fusion feature based on the scale factor; the scale factor is obtained by training the classifier in the data classification model;

[0229] The category determination unit 134 is used to perform key recognition on features corresponding to key features in the scaled fused features based on a classifier, and classify the scaled fused features based on the key recognition results to obtain the image category to which the target detection image belongs.

[0230] In terms of classifying the fused features based on the key features to obtain the image category to which the target detection image belongs, the data classification module 13 includes:

[0231] A result acquisition unit 135 is used to classify the fused features based on the key features to obtain at least two predicted labels and a predicted probability value of each predicted label;

[0232] The category determination unit 134 is used to determine the prediction label with the largest prediction probability value as the image category to which the target detection image belongs.

[0233] The device 800 further includes:

[0234] The communication acquisition module 14 is used to acquire the communication method associated with the image category if the image category belongs to the image abnormality category;

[0235] The message sending module 15 is used to send an object abnormality message to the terminal device based on the communication method, so that the terminal device detects the object corresponding to the original image based on the object abnormality message; the object abnormality message includes the image category.

[0236] The embodiment of the present application provides a data classification device, which obtains a visual identifier corresponding to an original image, identifies the detection position coordinates used to extract the object of interest in the original image, obtains a target detection image containing the object of interest from the original image based on the detection position coordinates, and determines the detection position information corresponding to the original image according to the visual identifier and the detection position coordinates; the visual identifier is used to represent the visual angle of the object of interest in the object corresponding to the original image; obtains the image features of the target detection image, scales the detection position information, and obtains the position features of the target detection image; performs feature splicing on the image features and the position features to obtain fusion features, determines the key features in the image features based on the position features, and classifies the fusion features based on the key features to obtain the image category to which the target detection image belongs. By combining the target detection image and the detection position information of the target detection image in the original image, it is possible to realize the recognition and classification of the target detection image based on the position information, so that under different position information, the recognition emphasis of the detection image is different, and the position corresponding to the acquired image category can be obtained, thereby improving the accuracy of data recognition and classification.

[0237] For further information, see Fig. 9 , Fig. 9 Schematic diagram of another data classification device provided in an embodiment of the present application. The data classification device may be a computer program (including program code, etc.) running in a computer device, for example, the data classification device may be an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig. 9 As shown, the data classification device 900 can be used for Figure 7 The computer device in the corresponding embodiment, specifically, the device may include: a sample acquisition module 16, a sample feature acquisition module 17, a sample classification module 18 and a model training module 19.

[0238] The sample acquisition module 16 is used to acquire the detection image sample, the sample visual identification of the detection image sample, the detection sample position coordinates and the target sample label, and determine the detection sample position information of the detection image sample according to the sample visual identification and the detection sample position coordinates; the sample visual identification is used to indicate the visual angle of the detection image sample in the object corresponding to the detection image sample;

[0239] The sample feature acquisition module 17 is used to use the initial convolutional neural network in the initial data classification model to obtain the sample image features of the detection image samples, and use the initial multi-layer perceptron in the initial data classification model to scale the detection sample position information to obtain the sample position features of the detection image samples;

[0240] The sample classification module 18 is used to perform feature splicing on the sample image features and the sample position features to obtain sample fusion features, determine the sample key features of the sample image features based on the sample position features, and classify the sample fusion features based on the sample key features to obtain the image sample category to which the detection image sample belongs;

[0241] The model training module 19 is used to train the initial data classification model based on the loss function between the image sample category and the target sample label to obtain the data classification model.

[0242] In terms of classifying the sample fusion features based on the sample key features to obtain the image sample category to which the detected image sample belongs, the sample classification module 18 includes:

[0243] The probability acquisition unit 181 is used to classify the sample fusion features based on the sample key features to obtain at least two sample labels and a sample prediction probability value of each sample label; the at least two sample labels include a target sample label;

[0244] The sample category determination unit 182 is used to determine the sample label with the largest sample prediction probability value as the image sample category to which the detection image sample belongs.

[0245] Wherein, the loss function includes a first loss function and a second loss function;

[0246] The model training module 19 includes:

[0247] A first loss generating unit 191, configured to generate a first loss function according to the image sample category and the target sample label;

[0248] A second loss generating unit 192 is used to generate a label distribution function according to the target sample label, generate a prediction distribution function according to at least two sample labels and the sample prediction probability value of each sample label, and generate a second loss function according to the label distribution function and the prediction distribution function;

[0249] The model generation unit 193 is used to train the initial data classification model according to the first loss function and the second loss function to obtain a data classification model.

[0250] An embodiment of the present application provides a data classification device, which can train a data classification model in combination with classification learning and distribution learning. The classification learning can truly reflect the predicted category confidence of the data classification model, and the distribution learning can ensure that the model can be trained smoothly, thereby improving the training reliability of the data classification model.

[0251] See also Fig.10 , Fig.10Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.10 As shown, the computer device in the embodiment of the present application may include: one or more processors 1001, a memory 1002, and an input-output interface 1003. The processor 1001, the memory 1002, and the input-output interface 1003 are connected via a bus 1004. The memory 1002 is used to store a computer program, which includes program instructions, and the input-output interface 1003 is used to receive and output data, such as for data exchange between the computer device and the terminal device; the processor 1001 is used to execute the program instructions stored in the memory 1002.

[0252] The processor 1001 is used to perform the following operations when making predictions through the data classification model:

[0253] Obtaining a visual identifier corresponding to the original image, identifying the detection position coordinates for extracting the object of interest in the original image, obtaining a target detection image containing the object of interest from the original image based on the detection position coordinates, and determining the detection position information corresponding to the original image based on the visual identifier and the detection position coordinates; the visual identifier is used to indicate the visual angle of the object of interest in the object corresponding to the original image;

[0254] Acquire the image features of the target detection image, perform scale transformation on the detection position information, and obtain the position features of the target detection image;

[0255] The image features and position features are concatenated to obtain fused features. The key features in the image features are determined based on the position features. The fused features are classified based on the key features to obtain the image category to which the target detection image belongs.

[0256] The processor 1001 is used to perform the following operations when training the data classification model:

[0257] Obtain a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, and determine the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate; the sample visual identifier is used to indicate the visual angle of the detection image sample in the object corresponding to the detection image sample;

[0258] The initial convolutional neural network in the initial data classification model is used to obtain sample image features of the detection image samples, and the initial multi-layer perceptron in the initial data classification model is used to scale the detection sample position information to obtain sample position features of the detection image samples;

[0259] Perform feature splicing on sample image features and sample position features to obtain sample fusion features, determine sample key features of sample image features based on sample position features, classify sample fusion features based on sample key features, and obtain the image sample category to which the detected image sample belongs;

[0260] Based on the loss function between the image sample categories and the target sample labels, the initial data classification model is trained to obtain the data classification model.

[0261] In some feasible implementations, the processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0262] The memory 1002 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1001 and the input / output interface 1003. A portion of the memory 1002 may also include a non-volatile random access memory. For example, the memory 1002 may also store information about the device type.

[0263] In a specific implementation, the computer device can execute the following operations through its built-in functional modules: Figure 3 or Figure 7 For details on the implementation methods provided in each step, please refer to the Figure 3 or Figure 7 The implementation methods provided in each step will not be repeated here.

[0264] The embodiment of the present application provides a computer device, including: a processor, an input / output interface, and a memory, wherein the processor obtains a computer program in the memory and executes the computer program. Figure 3The various steps of the method shown in are used to perform data classification operations. The embodiment of the present application realizes obtaining the detection position information used to extract the object of interest in the original image, obtaining the target detection image containing the object of interest from the original image based on the detection position information; obtaining the image features of the target detection image, performing scale transformation on the detection position information, and obtaining the position features of the target detection image; performing feature splicing on the image features and the position features to obtain fusion features, and classifying the fusion features to obtain the image category to which the target detection image belongs. By combining the target detection image and the detection position information of the target detection image in the original image, it is possible to realize the recognition and classification of the target detection image with respect to the position information, so that under different position information, the recognition emphasis of the detection image is different, and the position corresponding to the acquired image category can be obtained, thereby improving the accuracy of data recognition and classification.

[0265] The embodiment of the present application provides a computer device, including: a processor, an input / output interface, and a memory, wherein the processor obtains a computer program in the memory and executes the computer program. Figure 7 The various steps of the method shown in the figure are used to perform the training operation of the data classification model. The embodiment of the present application realizes the training process of the data classification model, combines classification learning and distribution learning to train the data classification model, and truly reflects the predicted category confidence of the data classification model through classification learning. Through distribution learning, it is ensured that the model can be trained smoothly, thereby improving the training reliability of the data classification model.

[0266] The present application also provides a computer-readable storage medium storing a computer program suitable for being loaded and executed by the processor. Figure 3 For details on the data classification methods provided in each step, please refer to the Figure 3 Alternatively, the embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, the computer program being suitable for being loaded and executed by the processor. Figure 7 For details on the data classification methods provided in each step, please refer to the Figure 7 The implementation methods provided in each step are not repeated here. In addition, the description of the beneficial effects of adopting the same method is not repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer program can be deployed to execute on one computer device, or on multiple computer devices located in one place, or on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0267] The computer-readable storage medium may be the data classification device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0268] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 3 The method provided in the various optional methods realizes obtaining the channel feature map corresponding to the multimedia data based on the channel attention mechanism, processing the channel feature map based on the spatial attention mechanism, obtaining the spatial feature map of the multimedia data, determining the predicted media quality of the multimedia data according to the spatial feature map, and combining the channel attention mechanism and the spatial attention mechanism to realize the attention to the distorted and degraded areas in the multimedia data, thereby improving the accuracy of quality prediction. The quality prediction model in this application is based on end-to-end training, which can optimize the quality prediction model in the iterative process, while maintaining the simplicity of the convolutional attention module and improving the prediction speed.

[0269] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0270] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0271] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data classification device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data classification device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data classification device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data storage device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0272] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data classification method, characterized in that: The method comprises: Obtaining a visual identifier corresponding to an original image, identifying detection position coordinates for extracting an object of interest in the original image, obtaining a target detection image containing the object of interest from the original image based on the detection position coordinates, and determining detection position information corresponding to the original image according to the visual identifier and the detection position coordinates; the visual identifier is used to indicate a visual angle of the object of interest in an item corresponding to the original image; Acquire image features of the target detection image; The detection position information is scaled to obtain the position features of the target detection image; including: obtaining the image width and image height of the target detection image, normalizing the detection position coordinates based on the image width and the image height to obtain normalized position coordinates; obtaining the total number of visuals, normalizing the visual identifier based on the total number of visuals to obtain a normalized visual identifier; generating a perceptual input feature based on the normalized position coordinates and the normalized visual identifier, and using a multi-layer perceptron in a data classification model to scale the perceptual input feature to obtain the position features of the target detection image; The image features and the position features are concatenated to obtain fused features, key features in the image features are determined based on the position features, and the fused features are classified based on the key features to obtain the image category to which the target detection image belongs.

2. The method according to claim 1, characterized in that The step of identifying the detection position coordinates for extracting the object of interest in the original image, and acquiring a target detection image containing the object of interest from the original image based on the detection position coordinates, comprises: Acquire an original image, perform object detection on the original image, acquire a first predicted bounding box containing an object of interest, and determine detection position coordinates according to first position information of the first predicted bounding box; A second predicted border is determined according to the detection position coordinates, and an area indicated by the second predicted border in the original image is determined as a target detection image.

3. The method according to claim 2, characterized in that The determining the detection position coordinates according to the first position information of the first predicted border includes: Determining a border width and a border height of the first predicted border according to the first position information of the first predicted border; The first predicted border is resized based on the border width and the border height, and the detection position coordinates are determined according to the adjusted first predicted border.

4. The method according to claim 3, characterized in that The method further comprises: adjusting the size of the first predicted border based on the border width and the border height, and determining the detection position coordinates according to the adjusted first predicted border, including: According to the difference between the border width and the border height and the first position information, the first predicted border is resized to determine the second position information and the adjusted width and height of the adjusted first predicted border; Acquire a frame expansion coefficient, transform the size of the adjustment width and the adjustment height based on the frame expansion coefficient, and determine the detection position coordinates according to the second position information and the transformed adjustment width and adjustment height.

5. The method according to claim 1, characterized in that The acquiring the image features of the target detection image comprises: The target detection image is input into a convolutional neural network in a data classification model, and features of the target detection image are extracted based on a convolutional layer in the convolutional neural network to obtain image features of the target detection image.

6. The method according to claim 1, characterized in that The step of performing feature stitching on the image features and the position features to obtain fusion features includes: Normalizing the image features to obtain normalized image features, and normalizing the position features to obtain normalized position features; The normalized image features and the normalized position features are concatenated to obtain fused features.

7. The method according to claim 1, characterized in that The classifying process of the fused features based on the key features to obtain the image category to which the target detection image belongs includes: Acquire a scale factor in a classifier of a data classification model, and scale transform the fusion feature based on the scale factor; the scale factor is obtained by training the classifier in the data classification model; Based on the classifier, key recognition is performed on features corresponding to the key features in the scale-transformed fusion features, and based on the key recognition results, classification processing is performed on the scale-transformed fusion features to obtain the image category to which the target detection image belongs.

8. The method according to claim 1, characterized in that The classifying process of the fused features based on the key features to obtain the image category to which the target detection image belongs includes: Classify the fused features based on the key features to obtain at least two prediction labels and a prediction probability value for each prediction label; The predicted label with the largest predicted probability value is determined as the image category to which the target detection image belongs.

9. The method according to claim 1, characterized in that The method further comprises: If the image category belongs to the image abnormality category, obtaining the communication method associated with the image category; An object abnormality message is sent to a terminal device based on the communication method, so that the terminal device detects the object corresponding to the original image based on the object abnormality message; the object abnormality message includes the image category.

10. A data classification method, characterized in that: The method comprises: Acquire a detection image sample, a sample visual identifier of the detection image sample, a detection sample position coordinate, and a target sample label, wherein the detection sample position coordinate is a position coordinate used to extract an object of interest in the detection image sample, and determine the detection sample position information of the detection image sample according to the sample visual identifier and the detection sample position coordinate; the sample visual identifier is used to indicate a visual angle of the object of interest in an object corresponding to the detection image sample; Using an initial convolutional neural network in an initial data classification model to obtain sample image features of the detection image sample; The initial multi-layer perceptron in the initial data classification model is used to scale the detection sample position information to obtain the sample position features of the detection image sample; including: obtaining the image width and image height of the detection image sample, normalizing the detection sample position coordinates based on the image width and the image height to obtain normalized sample position coordinates; obtaining the total number of sample visuals, normalizing the sample visual identifier based on the total number of sample visuals to obtain a normalized sample visual identifier; generating initial perceptual input features based on the normalized sample position coordinates and the normalized sample visual identifier, and scaling the initial perceptual input features using the initial multi-layer perceptron in the initial data classification model to obtain the sample position features of the detection image sample; Performing feature splicing on the sample image feature and the sample position feature to obtain a sample fusion feature, determining a sample key feature of the sample image feature based on the sample position feature, and classifying the sample fusion feature based on the sample key feature to obtain an image sample category to which the detection image sample belongs; Based on the loss function between the image sample category and the target sample label, the initial data classification model is trained to obtain a data classification model.

11. The method according to claim 10, characterized in that The classifying process of the sample fusion feature based on the sample key feature to obtain the image sample category to which the detection image sample belongs includes: Classify the sample fusion features based on the sample key features to obtain at least two sample labels and a sample prediction probability value of each sample label; the at least two sample labels include the target sample label; The sample label having the largest sample prediction probability value is determined as the image sample category to which the detection image sample belongs.

12. The method according to claim 11, characterized in that The loss function includes a first loss function and a second loss function; The training of the initial data classification model based on the loss function between the image sample category and the target sample label to obtain the data classification model includes: Generate the first loss function according to the image sample category and the target sample label; Generate a label distribution function according to the target sample label, generate a prediction distribution function according to the at least two sample labels and the sample prediction probability value of each sample label, and generate the second loss function according to the label distribution function and the prediction distribution function; The initial data classification model is trained according to the first loss function and the second loss function to obtain a data classification model.

13. A computer device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store a computer program, and the processor is used to call the computer program to execute the method described in any one of claims 1 to 9, or execute the method described in any one of claims 10 to 12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 9, or executing the method according to any one of claims 10 to 12.

Images