Data processing method, device, equipment, readable storage medium and program product

Abstract

The present application discloses a data processing method, device, equipment, readable storage medium and program product, which relates to deep neural network learning technology in the multimedia field. The method includes: obtaining media data features corresponding to object media data; using the first channel attention parameter to perform feature extraction on the media data features to obtain the first type of object features corresponding to the media data features; using the second channel attention parameter to perform feature extraction on the media data features to obtain the second type of object features corresponding to the media data features; performing feature recognition on the first type of object features to obtain the position information of the first type of object in the object media data; performing feature recognition on the second type of object features to obtain the position information of the second type of object in the object media data. By using the present application, the different position information of the first type of object and the second type of object can be more accurately identified, and the accuracy of separating the position information of the object media data can be improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a data processing method, device, equipment, readable storage medium and program product. Background Art

[0002] With the advent of the digital age, the separation object media data location information technology will be applied in more and more occasions, and people have higher and higher requirements for the accuracy of the separation object media data location information. The separation object media data location information is generally based on deep learning to perform location information separation and other processing. Common location information separation technologies rely on layer separation or color separation to achieve location information separation.

[0003] Among them, the separation of position information by color is easily affected by lighting, occlusion and overlap between characters, etc., which may lead to errors in the separation of position information and incomplete output of the final structured information. Based on layer separation, different types of objects in the object media data will be separated into corresponding images for recognition. In this way, the learning of image features and the distribution of generated type objects are relatively local, loose and unstable. In other words, the above method may reduce the accuracy of position information separation and recognition. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device, readable storage medium and program product, which can more accurately identify the different location information of a first type of object and a second type of object, and improve the accuracy of the location information of separated object media data (such as medical bill data, etc.).

[0005] On the one hand, an embodiment of the present application provides a data processing method, including:

[0006] Obtaining media data features corresponding to the object media data;

[0007] Using the first channel attention parameter to extract the media data feature, to obtain the first type of object feature corresponding to the media data feature;

[0008] The second channel attention parameter is used to extract the feature of the media data to obtain the second type of object feature corresponding to the media data feature; the first channel attention parameter and the second channel attention parameter are obtained by training under different object types;

[0009] Feature recognition is performed on the first type of object features to obtain location information of the first type of object in the object media data; feature recognition is performed on the second type of object features to obtain location information of the second type of object in the object media data.

[0010] On the one hand, an embodiment of the present application provides another data processing method, including:

[0011] Obtaining a first sample label corresponding to a first type of object in the sample media data, and obtaining a second sample label corresponding to a second type of object in the sample media data;

[0012] Inputting the sample media data into the initial position recognition model, and obtaining sample data features corresponding to the sample media data in the initial position recognition model;

[0013] In the initial position recognition model, a first initial channel attention parameter is used to extract features of the sample data to obtain a first type of sample feature corresponding to the sample data feature;

[0014] In the initial position recognition model, the second initial channel attention parameter is used to extract the sample data features to obtain the second type of sample features corresponding to the media data features;

[0015] In a first type recognition network of the initial position recognition model, feature recognition is performed on first type sample features to obtain position information of first type objects in sample media data; in a second type recognition network of the initial position recognition model, feature recognition is performed on second type sample features to obtain position information of second type objects in sample media data;

[0016] According to the position information and the first sample label of the first type of object in the sample media data, as well as the position information and the second sample label of the second type of object in the sample media data, the first initial channel attention parameters and the second initial channel attention parameters in the initial position recognition model are adjusted to obtain the first channel attention parameters, the second channel attention parameters, and the position recognition model including the first channel attention parameters and the second channel attention parameters.

[0017] An embodiment of the present application provides a data processing device, including:

[0018] An object feature acquisition module, used to acquire media data features corresponding to the object media data;

[0019] A first feature extraction module, configured to extract features of the media data using a first channel attention parameter to obtain first type object features corresponding to the media data features;

[0020] A second feature extraction module is used to extract features of the media data using the second channel attention parameter to obtain a second type of object feature corresponding to the media data feature; the first channel attention parameter and the second channel attention parameter are obtained by training under different object types;

[0021] The first feature recognition module is used to perform feature recognition on the first type of object features to obtain the position information of the first type of object in the object media data; input the second type of object features into the position recognition model, perform feature recognition, and obtain the position information of the second type of object in the object media data.

[0022] The data processing device further includes:

[0023] A semantic parsing module is used to obtain initial media data, perform semantic parsing on the initial media data, and obtain content direction information in the initial media data;

[0024] The direction adjustment module is used to adjust the direction of the initial media data based on the content direction information to obtain the object media data.

[0025] Among them, the semantic parsing module includes:

[0026] An activation unit, used to perform convolution and activation processing on the initial media data to obtain preliminary activation features corresponding to the initial media data;

[0027] A second conversion unit is used to perform feature conversion processing on the preliminary activation feature based on the hourglass layer to obtain a media adjustment feature;

[0028] The pooling unit is used to perform pooling processing on the media adjustment features to obtain pooling features, and perform full-connection prediction processing on the pooling features to obtain content direction information in the initial media data.

[0029] Among them, the object feature acquisition module includes:

[0030] An acquisition unit, used to acquire object media data, perform feature extraction processing on the object media data, and obtain initial media features of the object media data;

[0031] The first conversion unit is used to perform feature conversion on the initial media feature to obtain the media data feature corresponding to the target media data.

[0032] Among them, the convolution unit includes:

[0033] The convolution subunit is used to perform multi-layer convolution on the initial media features to obtain N convolution media features; N is a positive integer;

[0034] The upsampling fusion subunit is used to perform upsampling feature fusion processing on N convolutional media features to obtain convolutional fusion features;

[0035] The feature fusion subunit is used to perform feature fusion processing on the N convolutional media features and the convolutional fusion features to obtain media data features corresponding to the object media data.

[0036] Wherein, the first feature recognition module includes:

[0037] A first preprocessing unit, configured to perform feature preprocessing on a first type of object feature to obtain a first probability distribution map and a first threshold mapping map corresponding to the first type of object in the object media data;

[0038] A first mapping unit, configured to perform binarization processing on the first probability distribution map and the first threshold mapping map to obtain a first approximate binary map corresponding to the first type of object in the object media data;

[0039] A first determining unit, configured to determine position information of a first type of object in the object media data according to the first approximate binary image;

[0040] A second preprocessing unit, configured to perform feature preprocessing on the second type of object features to obtain a second probability distribution map and a second threshold mapping map corresponding to the second type of object in the object media data;

[0041] A second mapping unit, configured to perform binarization processing on the second probability distribution map and the second threshold mapping map to obtain a second approximate binary map corresponding to the second type of object in the object media data;

[0042] The second determining unit is used to determine the position information of the second type object in the object media data according to the second approximate binary image.

[0043] An embodiment of the present application provides a data processing device, including:

[0044] A sample label acquisition module, used to acquire a first sample label corresponding to a first type of object in sample media data, and acquire a second sample label corresponding to a second type of object in sample media data;

[0045] A sample feature acquisition module, used to input the sample media data into the initial position recognition model, and obtain the sample data features corresponding to the sample media data in the initial position recognition model;

[0046] A third feature extraction module is used to extract features of the sample data using the first initial channel attention parameter in the initial position recognition model to obtain a first type of sample feature corresponding to the sample data feature;

[0047] A fourth feature extraction module, used to extract features of the sample data using the second initial channel attention parameter in the initial position recognition model to obtain a second type of sample feature corresponding to the media data feature;

[0048] A second feature recognition module is used to perform feature recognition on a first type of sample feature in a first type recognition network of an initial position recognition model to obtain position information of a first type of object in sample media data; and to perform feature recognition on a second type of sample feature in a second type recognition network of an initial position recognition model to obtain position information of a second type of object in sample media data;

[0049] The model determination module is used to adjust the parameters of the initial position recognition model according to the position information of the first type of object in the sample media data and the first sample label, and the position information of the second type of object in the sample media data and the second sample label, to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0050] The model determination module includes:

[0051] A first function acquisition unit, configured to obtain a first sample loss function according to position information of a first type of object in sample media data and a first sample label;

[0052] A second function acquisition unit, configured to obtain a second sample loss function according to position information of the second type of object in the sample media data and a second sample label;

[0053] A balancing function acquisition unit, configured to obtain a sample balancing loss function according to a correlation between position information of a first type of object in the sample media data and position information of a second type of object in the sample media data;

[0054] The model determination unit is used to adjust the parameters of the initial position recognition model according to the first sample loss function, the second sample loss function and the sample balance loss function to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0055] On one hand, the present application provides a computer device, including: a processor, a memory, and a network interface;

[0056] The above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication function, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program so that the computer device executes the method in the embodiment of the present application.

[0057] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.

[0058] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the embodiment of the present application.

[0059] In an embodiment of the present application, the media data features corresponding to the object media data are obtained; the first channel attention parameter is used to extract the features of the media data features to obtain the first type of object features corresponding to the media data features; the second channel attention parameter is used to extract the features of the media data features to obtain the second type of object features corresponding to the media data features; the first channel attention parameter and the second channel attention parameter are obtained by training under different object types; the first type of object features are feature recognized to obtain the position information of the first type of object in the object media data; the second type of object features are feature recognized to obtain the position information of the second type of object in the object media data. The embodiment of the present application introduces the first channel attention parameter and the second channel attention parameter, and feature extraction is performed through different channel attention parameters to identify the position information of different types of objects in the object media data (such as medical bill data, etc.), that is, the feature distinction can be made more clearly. By performing feature recognition on two different types of objects, the position information of the first type of object and the position information of the second type of object can be more accurately identified according to different feature characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0062] Figure 2 is a schematic diagram of a scenario for separating location information provided by an embodiment of the present application;

[0063] Figure 3a It is a flowchart of a data processing method provided in an embodiment of the present application;

[0064] Figure 3b is a structural diagram for obtaining object media data provided by an embodiment of the present application;

[0065] Figure 4 It is a structural diagram for separating location information provided by an embodiment of the present application;

[0066] Figure 5 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0067] Figure 6 is a schematic diagram of the structure of sample media data provided in an embodiment of the present application;

[0068] Figure 7 is a structural diagram of an initial position recognition model provided by an embodiment of the present application;

[0069] Figure 8a It is a data structure diagram of a basic balanced loss function provided in an embodiment of the present application;

[0070] Figure 8b It is a flow chart of a process for separating location information provided by an embodiment of the present application;

[0071] Figure 8c It is a flow chart of a test location information separation process provided by an embodiment of the present application;

[0072] Figure 9a is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0073] Figure 9b is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0074] Fig.10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0076] Among them, if it is necessary to collect media data of an object (such as a user, etc.) in this application, a prompt interface or pop-up window will be displayed before or during the collection. The prompt interface or pop-up window is used to prompt the user that XXXX data is currently being collected. Only after the user confirms the prompt interface or pop-up window, the relevant steps for data acquisition will be started, otherwise the process will end. Moreover, the acquired user media data will be used in reasonable and legal scenarios or purposes. Optionally, in some scenarios where user media data needs to be used but has not been authorized by the user, authorization can be requested from the user, and the user media data can be used when the authorization is passed.

[0077] Among them, the present application relates to deep learning technology in the field of artificial intelligence. Through this deep learning technology, the prediction of position information between a first type of object and a second type of object and the training of a position recognition model are realized.

[0078] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0079] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0080] Machine Learning is a discipline that studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0081] Deep learning (DL) is a new research direction in the field of machine learning (ML). It is introduced into machine learning to make it closer to its original goal - artificial intelligence (AI). Deep learning is the inherent laws and representation levels of learning sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, and other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes great progress in artificial intelligence related technologies.

[0082] See also Figure 1 , Figure 1 Schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 As shown, the system may include a computer device 100 and a terminal cluster, and the terminal cluster may include: terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. It is understandable that the above system may include one or more terminal devices, and the present application does not limit the number of terminal devices. Among them, the terminal device mentioned above may be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a laptop computer, a PDA, a vehicle-mounted device, an augmented reality / virtual reality (Augmented Reality / Virtual Reality, AR / VR) device, a helmet display, a smart TV, a wearable device, a smart speaker, a digital camera, a camera and other mobile Internet devices (mobile internet device, MID) with network access capability, or terminal devices in scenarios such as trains, ships, and flights.

[0083] The computer device mentioned in this application may be a server or a terminal device, or a system composed of a server and a terminal device.

[0084] Among them, there can be a communication connection between terminal clusters, for example, there is a communication connection between terminal device 200a and terminal device 200b, and there is a communication connection between terminal device 200a and terminal device 200c. At the same time, any terminal device in the terminal cluster can have a communication connection with the business server 100, for example, there is a communication connection between terminal device 200a and business server 100, wherein the above communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, and this application does not limit it here.

[0085] It should be understood that Figure 1 Each terminal device in the terminal cluster shown can be any terminal device integrated with a function of identifying object media data, or a computer device integrated with a function of identifying object media data.

[0086] The object media data refers to data containing multiple object types on the same page, which may be data related to the user, such as the user's invoice information (such as taxi invoice information), medical bill data, user information storage address, etc. The computer device may obtain the object media data to be separated from any terminal device or the computer device itself, and separate the location information of the object media data, or obtain sample object media data for model training from any terminal device or the computer device itself, and perform model training based on the obtained sample object media data to obtain a location recognition model.

[0087] For the convenience of subsequent understanding and explanation, please refer to Figure 2 , Figure 2 is a schematic diagram of a scenario for separating location information provided by an embodiment of the present application. Figure 2In the embodiment, the computer device 300 can obtain the media data features corresponding to the object media data; use the first channel attention parameter to extract the features of the media data to obtain the first type of object features corresponding to the media data features; use the second channel attention parameter to extract the features of the media data to obtain the second type of object features corresponding to the media data features; perform feature recognition on the first type of object features to obtain the position information of the first type of object in the object media data; perform feature recognition on the second type of object features to obtain the position information of the second type of object in the object media data. Specifically, the position recognition model can be obtained through model training, and the computer device 300 can obtain the initial media data and the sample media data, and obtain the initial media features of the initial media data and the initial sample features of the sample media data. The computer device 300 can perform convolution processing and upsampling processing in the initial position recognition model on the initial media features to obtain the media data features. The computer device 300 can use the first channel attention parameter to extract the features of the media data to obtain the first type of object features corresponding to the media data features. The computer device 300 can use the second channel attention parameter to extract the features of the media data to obtain the second type of object features corresponding to the media data features. The computer device 300 can perform feature preprocessing on the first type of object features to obtain a first probability distribution map and a first threshold map corresponding to the first type of object in the object media data; perform binarization processing on the first probability distribution map and the first threshold map to obtain a first approximate binary map corresponding to the first type of object in the object media data; and determine the position information of the first type of object in the object media data based on the first approximate binary map. The computer device 300 can perform feature preprocessing on the second type of object features to obtain a second probability distribution map and a second threshold map corresponding to the second type of object in the object media data; perform binarization processing on the second probability distribution map and the second threshold map to obtain a second approximate binary map corresponding to the second type of object in the object media data; and determine the position information of the second type of object in the object media data based on the second approximate binary map.

[0088] It is understandable that the method provided in the embodiments of the present application can be executed by a computer device, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. Among them, the terminal device and the computer device can be directly or indirectly connected by wire or wireless means, and the embodiments of the present application are not limited here.

[0089] It is understandable that the above system architecture can be applied to scenarios such as text detection systems, medical systems, and recognition and separation of various types of objects (such as recognition and separation of printed characters and printed characters). Specific business scenarios will not be listed one by one here.

[0090] For further information, see Figure 3a , Figure 3a is a flow chart of a data processing method provided in an embodiment of the present application. Figure 3a As shown, the data processing method may at least include the following steps S101 to S104.

[0091] Step S101: acquiring media data features corresponding to target media data.

[0092] Specifically, object media data may refer to data containing multiple object types on the same page. For example, data such as a user's invoice information, medical bill data, collection forms, and the user's information storage address include image data. For example, taking invoice information or medical bill data as an example, the object media data may include a first type of object and a second type of object. Optionally, the first type of object may be a printed character, and the second type of object may be a printed character; taking a collection form (such as a questionnaire, etc.) as an example, the object media data may include a first type of object and a second type of object. Optionally, the first type of object may be a printed character, and the second type of object may be a handwritten character, etc. The media data features corresponding to the object media data may be obtained by performing feature extraction processing on the object media data.

[0093] It should be understood that the object media data can be obtained after direction adjustment or directly obtained. Specifically, the initial media data can be obtained and determined as the object media data; or, the initial media data can be semantically parsed to obtain the content direction information in the initial media data; the initial media data can be directionally adjusted based on the content direction information to obtain the object media data.

[0094] Optionally, in a possible way of obtaining content direction information, the initial media data can be convolved and activated to obtain preliminary activation features corresponding to the initial media data. The preliminary activation features are subjected to feature conversion processing based on the hourglass layer to obtain media adjustment features. Among them, the media adjustment features can be used to represent preliminary detection information of the content (such as characters, etc.) in the initial media data, that is, the relevant features preliminarily extracted from the position and direction of the characters, etc. in the initial media data. The media adjustment features are pooled to obtain pooled features, and the pooled features are fully connected to predict and obtain content direction information in the initial media data. In other words, the media adjustment features are further subjected to feature extraction and conversion to obtain content direction information in the initial media data, and the content direction information is used to represent the direction of the content in the initial media data, such as the deviation from the standard content direction. For example, for medical bill data, if the content direction information of the medical bill data is 0 degrees, it means that the direction of the content in the medical bill data is the same as the standard content direction; if the content direction information of the medical bill data is 90 degrees clockwise, it means that the direction angle between the direction of the content in the medical bill data and the standard content direction is 90 degrees clockwise; if the content direction information of the medical bill data is 180 degrees clockwise, it means that the direction angle between the direction of the content in the medical bill data and the standard content direction is 180 degrees clockwise; if the content direction information of the medical bill data is 270 degrees clockwise, it means that the direction angle between the direction of the content in the medical bill data and the standard content direction is 270 degrees clockwise. The initial media data is adjusted in direction based on the content direction information to obtain the object media data.

[0095] For example, when the direction of the bill character of a medical bill data is 90 degrees clockwise with the standard character direction, that is, the content direction information is 90 degrees clockwise, the initial media data can be adjusted in direction through the content direction information, that is, the initial media data is rotated 90 degrees counterclockwise to obtain object media data with the same direction as the standard content (that is, the content direction information is 0 degrees). By adjusting the direction of the initial media data, the object media data can be separated at the same angle, reducing the need for directional generalization processing of the features in the object media data, thereby improving the efficiency of object separation and, to a certain extent, improving the accuracy of object separation.

[0096] Optionally, the hourglass layer may include a channel convolution layer, a depth convolution layer, a normalization layer and an activation layer. The preliminary activation feature is input into the channel convolution layer, and at least two channel convolution processes are performed according to the magnitude to obtain a first object feature. The first object feature is input into the depth convolution layer, and a depth convolution process is performed to obtain a second object feature. The second object feature is input into the normalization layer, and a normalization process is performed to obtain a third object feature. The third object feature is input into the activation layer, and an activation process is performed to obtain a media adjustment feature. Optionally, the hourglass layer may also be other architectures, such as directly including a convolution layer and an activation layer, such as inputting the initial activation feature into the convolution layer for convolution processing to obtain an object convolution feature, and inputting the object convolution feature into the activation layer for feature adjustment to obtain a media adjustment feature, etc., which is not limited here.

[0097] Please also see Figure 3b , Figure 3b Schematic diagram of a structure for obtaining object media data provided by an embodiment of the present application. Figure 3b As shown, the initial media data is convolved and activated to obtain preliminary activation features. The preliminary activation features are input into the hourglass layer for feature conversion to obtain media adjustment features. Optionally, the hourglass layer may include any one or more layers of a channel convolution layer, a depth convolution layer, a normalization layer, and an activation layer. The preliminary activation features are input into the channel convolution layer for channel convolution to obtain second object features. The second object features are input into the normalization layer for normalization to obtain third object features. The third object features are input into the activation layer for activation to obtain media adjustment features.

[0098] It should be understood that the media adjustment features obtained by the hourglass layer are pooled to obtain pooled features, and the pooled features are fully connected to predict the content direction information in the initial media data. The initial media data is directionally adjusted based on the content direction information to obtain the object media data.

[0099] The content direction information can be used to confirm the relationship between the direction of the content in the initial media data and the standard content direction when scanning the characters in the medical bill data. For example, in the medical bill data, it can be considered to confirm the relationship between the character direction in the medical bill data and the standard character direction (i.e., the standard content direction). The direction of the initial media data can be adjusted by the content direction information, and the direction of the entire medical bill data can be adjusted to obtain the target media data.

[0100] Furthermore, the object media data can be obtained, and feature extraction processing can be performed on the object media data to obtain the initial media features of the object media data; the initial media features are a feature mapping result of the object media data, that is, the object media data is mapped to the feature spectrum to obtain features that can represent the semantic information of the initial media data (such as color value information, texture information, and content meaning, etc.). In other words, it is a feature mapping of the object media data. If the object media data is an image, after feature extraction processing is performed on the object media data, the image is mapped as a feature, and the initial media features are equivalent to a feature spectrum for the object media data. Feature conversion is performed on the initial media features to obtain the media data features corresponding to the object media data.

[0101] Specifically, multiple layers of convolution may be performed on the initial media features to obtain N convolution media features; N is a positive integer; upsampling feature fusion processing may be performed on the N convolution media features to obtain convolution fusion features; feature fusion processing may be performed on the N convolution media features and the convolution fusion features to obtain media data features corresponding to the object media data.

[0102] Please also see Figure 4 , Figure 4 Schematic diagram of a structure for separating location information provided by an embodiment of the present application. Figure 4 As shown, the initial media features of the object media data 400 can be obtained by convolving the initial media features of the object media data 400 with N convolution kernels to obtain N convolution media features, such as convolution media feature 401, convolution media feature 401a, convolution media feature 401b, and convolution media feature 401c. The convolution media features can be upsampled, and the upsampled convolution media features are fused with adjacent convolution media features to obtain convolution fusion features. Figure 4 The value of N is 4, but other values ​​of N are not limited) Figure 4 The up-sampled features are fused to obtain the convolutional fusion features, as shown in Figure 4 Specifically, N convolutional media features can be grouped into M feature pairs, where M is a positive integer less than N. Optionally, each feature pair includes two different convolutional media features. For example, two adjacent convolutional media features in the N convolutional media features can be grouped into a feature pair, such as Figure 4The feature pairs shown in may include a feature pair consisting of convolutional media feature 401b and convolutional media feature 401c, a feature pair consisting of convolutional media feature 401a and convolutional media feature 401b, and a feature pair consisting of convolutional media feature 401 and convolutional media feature 401a, that is, the convolutional media features included in the feature pair are adjacent; or, a feature pair consisting of two convolutional media features with a gap among the N convolutional media features, such as convolutional media feature 401a and convolutional media feature 401c, may be formed into a feature pair, and there is no limitation here.

[0103] Specifically, taking a feature pair as an example, the convolution sizes corresponding to the first convolution media feature and the second convolution media feature included in the feature pair can be obtained, and the first sampling parameter of the feature pair can be determined based on the convolution size of the first convolution media feature and the convolution size of the second convolution media feature. The first convolution media feature is upsampled based on the sampling parameter to obtain the first sampling feature of the first convolution media feature, and the first sampling feature and the second convolution media feature are subjected to feature fusion processing to obtain the convolution fusion feature corresponding to the feature pair, wherein the convolution size of the first convolution media feature is smaller than the convolution size of the second convolution media feature. Similarly, the convolution fusion features corresponding to M feature pairs can be obtained, that is, the number of the convolution fusion features can be considered to be M. Among them, "⊕" 402 refers to feature fusion processing.

[0104] Optionally, the Nth convolutional media feature among the N convolutional media features may be upsampled and fused with the (N-1)th convolutional media feature to obtain the first convolutional fusion feature. Specifically, the i-th convolutional media feature among the N convolutional media features may be upsampled and fused with the (Ni)th convolutional fusion feature to obtain the (N-i+1)th convolutional fusion feature, until i is a default threshold, and M convolutional fusion features are obtained. The default threshold can be set as needed, such as 2 or 3. i is a positive integer less than or equal to N. Specifically, Figure 4 As shown, the convolution media feature 401c is upsampled to obtain an upsampled feature, the convolution media feature 401b is feature fused with the upsampled feature of the convolution media feature 401c to obtain a convolution fusion feature 403a; the convolution fusion feature 403a is upsampled to obtain an upsampled feature of the convolution fusion feature 403a, and the upsampled feature of the convolution fusion feature 403a is feature fused with the convolution media feature 401a to obtain a convolution fusion feature 403. In this example, the M convolution fusion features include the convolution fusion feature 403 and the convolution fusion feature 403a.

[0105] Among them, the above examples are several processes of upsampling feature fusion processing of N convolutional media features, which do not limit other fusion methods. For example, interval fusion can be performed based on convolutional fusion features and convolutional media features, such as Figure 4 After the convolution fusion feature 403a in is upsampled, it is fused with the convolution media feature 401, etc., which is not limited here.

[0106] The N convolutional media features and convolutional fusion features are processed as follows: Figure 4 The feature fusion processing shown in Figure 4 In the embodiment, the convolution media feature and the convolution fusion feature are subjected to feature fusion through the feature fusion module 404, and the media data feature corresponding to the object media data can be obtained. The second sampling parameter corresponding to each convolution fusion feature can be obtained, and based on the second sampling parameter corresponding to the convolution fusion feature, the convolution fusion feature is upsampled to obtain the second sampling feature, that is, the second sampling features corresponding to the M convolution fusion features can be obtained. Optionally, if the convolution size of the convolution fusion feature is the largest among the M convolution fusion features, the second sampling parameter of the convolution fusion feature can be determined to be 1, that is, the convolution fusion feature can be not upsampled. The M second sampling features are fused to obtain the media data feature corresponding to the object media data.

[0107] Optionally, a third convolution media feature with the smallest convolution size can be obtained from the N convolution media features, and a third sampling parameter of the third convolution media feature is determined based on the convolution sizes corresponding to the M convolution fusion features, and the third convolution media feature is upsampled to obtain a third sampling feature, wherein the second sampling features corresponding to the M convolution fusion features can be obtained. The M second sampling features are subjected to feature fusion processing with the third sampling feature to obtain a media data feature corresponding to the object media data. For example, assuming that the convolution size of the convolution media feature 401c is 1 / 16, the convolution size of the convolution fusion feature 403a is 1 / 8, and the convolution size of the convolution fusion feature 403 is 1 / 4, then based on the convolution size, it can be determined that the third sampling parameter of the convolution media feature 401c is 4, the second sampling parameter of the convolution fusion feature 403a is 2, and the second sampling parameter of the convolution fusion feature 403 is 1, then based on the third sampling parameter of the convolution media feature 401c, the convolution media feature 401c is upsampled to obtain the upsampled features of the convolution media feature 401c; based on the convolution size, The convolution fusion feature 403a is upsampled based on the second sampling parameter of the convolution fusion feature 403a to obtain the upsampled feature of the convolution fusion feature 403a; the convolution fusion feature 403 is upsampled based on the second sampling parameter of the convolution fusion feature 403 to obtain the upsampled feature of the convolution fusion feature 403, and the upsampled feature of the convolution media feature 401c, the upsampled feature of the convolution fusion feature 403a and the upsampled feature of the convolution fusion feature 403 are fused through the feature fusion module 404 to obtain the media data feature corresponding to the object media data 400. Through multiple layers of convolution and upsampling, higher-level semantic features can be obtained. The convolution kernel size of each layer of convolution is different, and the implicit features of the object media data obtained are also different, so the implicit features of the object media data can be obtained in a more detailed manner. By extracting and fusing features at different levels and dimensions, the obtained media data features can represent the features in the media data of the object more comprehensively and richly. In other words, media data features containing more information can be obtained, so that in the subsequent object separation process, the features referenced and used are richer, thereby improving the accuracy of object separation.

[0108] It should be understood that the processes of performing multi-layer convolution on the initial media features, performing upsampling feature fusion processing on N convolution media features, and performing feature fusion processing on N convolution media features and convolution fusion features can be implemented by MobileNetV3 and ResNet-18 structures. Optionally, the processes of performing multi-layer convolution on the initial media features, performing upsampling feature fusion processing on N convolution media features, and performing feature fusion processing on N convolution media features and convolution fusion features can also be implemented by DenseNet, ResNet-34, ResNet-50 and ResNet-101 network structures.

[0109] Step S102: extracting features of the media data using the first channel attention parameter to obtain first type object features corresponding to the media data features.

[0110] Specifically, the first channel attention parameter is obtained by training the first type of object. Figure 4 In the embodiment, the first type of object feature 405 can be obtained from the feature fusion module based on the first channel attention parameter. The first channel attention parameter can be a parameter for the channel attention convolution (Squeeze-and-Excitation-Convolution, SE-Conv) module. Specifically, it can be considered that the media data feature includes media channel features under k channels, that is, k media channel features constitute the media data feature, and k is a positive integer. Among them, there are certain feature differences between the first type of object and the second type of object in terms of color and font. The feature difference will cause a part of the k channels to be strongly associated with the first type of object, and a part of the channels to be strongly associated with the second type of object, and the channels strongly associated with the first type of object are not completely the same as the channels strongly associated with the second type of object. Among them, the first channel attention parameter can be used to represent the first channel weight for the k media channel features. The first channel attention parameter is a parameter trained for the first type of object. It can be considered that the first channel weight of the channel strongly associated with the first type of object in the first channel attention parameter is greater than the first channel weight of other channels. By extracting features of the k media channel features through the first channel attention parameter, the features related to the first type of object can be strengthened, and the features that are irrelevant to or less associated with the first type of object can be weakened, thereby extracting the first type of object features corresponding to the first type of object in the media data features. For example, the first type of object is a printed character, and the second type of object is a printed character. The first channel attention parameter can be used to extract the first type of object features associated with the printed character from the media data features.

[0111] Optionally, the number of the first channel attention parameters can be r, where r is a positive integer, that is, r SE-Conv modules can be added to the first recognition module, each SE-Conv module includes a first channel attention parameter, and the media data feature is processed by r SE-Conv modules in sequence to obtain the first type of object feature corresponding to the media data feature, that is, the r first channel attention parameters are used to extract the features of the media data feature in sequence to obtain the first type of object feature corresponding to the media data feature. Among them, the value of r can be set as needed, for example, when r is 1, the speed of feature extraction can be accelerated, that is, the efficiency of feature separation of the first type of object and the second type of object in the media data feature can be improved; when r is greater than 1, the features of the first type of object and the second type of object in the media data feature can be separated and adjusted multiple times to improve the accuracy of object feature separation. For example, when r is greater than or equal to 2, the jth first channel attention parameter is used to extract the attention feature extracted by the (j-1)th channel attention parameter to obtain the attention feature extracted by the jth first channel attention parameter, j is a positive integer, and j is less than or equal to r. When j is r, the attention feature extracted by the rth first channel attention parameter is obtained, and the attention feature extracted by the rth first channel attention parameter is determined as the first type object feature. When j is 1, the first first channel attention parameter is used to extract the media data feature, and the attention feature extracted by the first first channel attention parameter is obtained. Figure 4 As shown, assuming that r is 2, the first first channel attention parameter (i.e., j is 1) is used to extract the media data features, and the attention feature 417 extracted by the first first channel attention parameter is obtained. The second first channel attention parameter (i.e., j is 2) is used to extract the media data features, and the attention feature 405 extracted by the second first channel attention parameter is obtained. At this time, j is r, and the attention feature 405 is determined as a first type object feature.

[0112] Step S103, using the second channel attention parameter to extract the media data features to obtain the second type of object features corresponding to the media data features; the first channel attention parameter and the second channel attention parameter are obtained by training under different object types.

[0113] Specifically, the second channel attention parameter is obtained by training the second type of object. Figure 4In the embodiment, the second type of object feature 411 can be obtained from the feature fusion module based on the second channel attention parameter. The second channel attention parameter can be a parameter for channel attention convolution (Squeeze-and-ExcitationConvolution, SE-Conv). Specifically, it can be considered that the media data feature includes media channel features under k channels, that is, k media channel features constitute the media data feature, and k is a positive integer. Among them, there are certain feature differences between the first type of object and the second type of object in terms of color and font. The feature difference will cause a part of the k channels to be strongly associated with the first type of object, and a part of the channels to be strongly associated with the second type of object, and the channels strongly associated with the first type of object are not completely the same as the channels strongly associated with the second type of object. Among them, the second channel attention parameter can be used to represent the second channel weight for the k media channel features. The second channel attention parameter is a parameter trained for the second type of object. It can be considered that the second channel weight of the channel strongly associated with the second type of object in the second channel attention parameter is greater than the second channel weight of other channels. By extracting features of the k media channel features through the second channel attention parameter, the features related to the second type of object can be strengthened, and the features that are irrelevant to or less associated with the second type of object can be weakened, thereby extracting the second type of object features corresponding to the second type of object in the media data features. For example, the first type of object is a printed character, and the second type of object is a printed character. The second type of object features associated with the printed character can be extracted from the media data features through the second channel attention parameter.

[0114] Optionally, the number of the second channel attention parameters can be r, where r is a positive integer, that is, r SE-Conv modules can be added to the second recognition module, each SE-Conv module includes a second channel attention parameter, and the media data feature is processed by r SE-Conv modules in sequence to obtain the second type of object feature corresponding to the media data feature, that is, the r second channel attention parameters are used to extract the features of the media data feature in sequence to obtain the second type of object feature corresponding to the media data feature. Among them, the value of r can be set as needed, for example, when r is 1, the speed of feature extraction can be accelerated, that is, the efficiency of feature separation of the first type of object and the second type of object in the media data feature can be improved; when r is greater than 1, the features of the first type of object and the second type of object in the media data feature can be separated and adjusted multiple times to improve the accuracy of object feature separation. For example, when r is greater than or equal to 2, the t-th second channel attention parameter is used to extract the attention feature extracted by the (t-1)-th channel attention parameter to obtain the attention feature extracted by the t-th second channel attention parameter, t is a positive integer, and t is less than or equal to r. When t is r, the attention feature extracted by the rth second channel attention parameter is obtained, and the attention feature extracted by the rth second channel attention parameter is determined as the second type object feature. When t is 1, the first second channel attention parameter is used to extract the media data feature, and the attention feature extracted by the first second channel attention parameter is obtained. Figure 4 As shown, assuming that r is 2, the first second channel attention parameter (i.e., t is 1) is used to extract the media data features, and the attention feature 418 extracted by the first second channel attention parameter is obtained. The second second channel attention parameter (i.e., t is 2) is used to extract the media data features, and the attention feature 411 extracted by the second second channel attention parameter is obtained. At this time, t is r, and the attention feature 411 is determined as a second type object feature.

[0115] By adding the first channel attention parameter and the second channel attention parameter, the characteristic information of different colors, fonts, etc. between printed characters and printed characters can be distinguished, thereby improving the recall index (Recall) of printed characters and printed characters by 8% and the precision index (Precision) by 7%. Among them, the first channel attention parameter and the second channel attention parameter are obtained through training, and can be used to distinguish the features associated with the first type of object and the second type of object in the media data features, so as to achieve the purpose of separating the type objects in the object media data, such as separating the printed characters and printed characters in the medical bill data.

[0116] Step S104, performing feature recognition on the first type of object features to obtain location information of the first type of object in the object media data; performing feature recognition on the second type of object features to obtain location information of the second type of object in the object media data.

[0117] Specifically, the first recognition module can perform feature recognition on the first type of object features to obtain the location information of the first type of object in the object media data. The second recognition module can perform feature recognition on the second type of object features to obtain the location information of the second type of object in the object media data.

[0118] Feature preprocessing is performed on the first type object features to obtain a first probability distribution map and a first threshold mapping map corresponding to the first type object in the object media data.

[0119] The first probability distribution map and the first threshold mapping map are binarized to obtain a first approximate binary map corresponding to the first type of object in the object media data.

[0120] According to the first approximate binary image, position information of the first type of object in the object media data is determined.

[0121] Feature preprocessing is performed on the second type object features to obtain a second probability distribution map and a second threshold mapping map corresponding to the second type object in the object media data.

[0122] The second probability distribution map and the second threshold mapping map are binarized to obtain a second approximate binary map corresponding to the second type of object in the object media data.

[0123] According to the second approximate binary image, position information of the second type object in the object media data is determined.

[0124] Please also see Figure 4 ,exist Figure 4 In the first recognition module 410, the first channel attention parameter can be used to extract the features of the media data to obtain the first type of object features 405 corresponding to the media data features, and the first type of object features 405 can be feature preprocessed to obtain the first probability distribution map 406 and the first threshold mapping map 407 corresponding to the first type of object in the object media data, and the first probability distribution map 406 and the first threshold mapping map 407 are binarized to obtain the first approximate binary map 408 corresponding to the first type of object in the object media data 400, and the position information 409 of the first type of object in the object media data is determined according to the first approximate binary map 408. The first recognition module 410 can perform feature recognition on the first type of object features 405 to obtain the position information 409 of the first type of object in the object media data.

[0125] See also Figure 4 In the second recognition module 416, the second channel attention parameter can be used to extract the features of the media data to obtain the second type object features 411 corresponding to the media data features, and the second type object features 411 are feature preprocessed to obtain the second probability distribution map 412 and the second threshold mapping map 413 corresponding to the second type object in the object media data, and the second probability distribution map 412 and the second threshold mapping map 413 are binarized to obtain the second approximate binary map 414 corresponding to the second type object in the object media data 400, and the position information 415 of the second type object in the object media data is determined according to the second approximate binary map 414. The second recognition module 416 can perform feature recognition on the second type object features 411 to obtain the position information 415 of the second type object in the object media data.

[0126] Understandably, Figure 4 The overall architecture (i.e., including obtaining object media data 400; convolving the initial media features of the object media data 400 to obtain N convolutional media features, such as convolutional media feature 401, convolutional media feature 401a, convolutional media feature 401b, and convolutional media feature 401c; upsampling and convolutionally fusing the convolutional media features; performing feature fusion through feature fusion module 404; a first recognition module 410; and a second recognition module 416) constitutes a text detection character separation module.

[0127] In an embodiment of the present application, the media data features corresponding to the object media data are obtained; the first channel attention parameter is used to extract the features of the media data features to obtain the first type of object features corresponding to the media data features; the second channel attention parameter is used to extract the features of the media data features to obtain the second type of object features corresponding to the media data features; the first type of object features are feature identified to obtain the position information of the first type of object in the object media data; the second type of object features are feature identified to obtain the position information of the second type of object in the object media data. By adopting the present application, the difficulty of identifying the position information can be reduced, thereby reducing the difficulty of structuring the extraction of data content information. The present application can separate the printed characters on the medical bill data from the printed characters based on different media data features, combined with the media data features such as the color and font of the printed characters in the pin-type or template format on the medical bill data, thereby reducing the difficulty of character recognition caused by the overlap and occlusion of the printed characters, thereby reducing the difficulty of structuring the extraction of the content information of the medical bill data, and improving the complete medical bill data information. By adopting the present application, different location information of first type objects and second type objects can be more accurately identified, thereby improving the accuracy of location information of separated object media data (such as medical bill data, etc.).

[0128] For further information, see Figure 5 , Figure 5 is a flow chart of a data processing method provided in an embodiment of the present application. Figure 5 As shown, the data processing method may at least include the following steps S201 to S206.

[0129] Step S201: obtaining a first sample tag corresponding to a first type of object in sample media data, and obtaining a second sample tag corresponding to a second type of object in sample media data.

[0130] Specifically, the first sample label may be manually labeled information about the first type of object in the sample media data. The second sample label may be manually labeled information about the second type of object in the sample media data. The sample media data may be data including the media data of the first type of object and the media data of the second type of object. The sample media data may be invoices, express tickets, and other bills that need to be printed. Please refer to Figure 6 , Figure 6 is a schematic diagram of the structure of a sample media data provided by an embodiment of the present application. Figure 6 As shown, the first type of object can be Figure 6 The second type of object can be a printed font part as shown in Figure 6 The Print Fonts section shown in .

[0131] Step S202: input the sample media data into the initial position recognition model, and obtain the sample data features corresponding to the sample media data in the initial position recognition model.

[0132] For details, see Figure 7 , Figure 7 Schematic diagram of the structure of an initial position recognition model provided by an embodiment of the present application. Figure 7 As shown, the initial sample features of the sample media data 503 are obtained. The sample media data may refer to data containing multiple object types on the same page, such as the user's invoice information, medical bill data, collection forms, and user information storage address data. The sample media data carries color or font labels. The initial sample features may be a feature mapping result of the sample media data. That is, the sample media data is mapped to a feature map to obtain features that can represent the semantic information of the initial sample data (such as color value information, texture information, and content meaning, etc.). In other words, it is a feature mapping of the sample media data. If the sample media data is an image, after feature extraction processing is performed on the sample media data, the image is mapped to a feature. The initial sample features are equivalent to a feature map for the sample media data. N convolution kernels may be used to convolve the initial sample features of the sample media data 503, and N sample media features may be obtained, such as sample media feature 504, sample media feature 504a, sample media feature 504b, and sample media feature 504c. Convolution of N sample media features ( Figure 7 The value of N is 4, but other values ​​of N are not limited) Figure 7 The up-sampled feature fusion process shown in FIG. 1 is performed to obtain the sample fusion feature, such as Figure 7 Specifically, the N sample convolution media features can be combined into M sample feature pairs, where M is a positive integer less than N. Optionally, each sample feature pair includes two different sample convolution media features. For example, two adjacent sample convolution media features in the N sample convolution media features can be combined into one sample feature pair, such as Figure 7The sample feature pairs shown in may include a sample feature pair consisting of sample convolution media feature 504b and sample convolution media feature 504c, a sample feature pair consisting of sample convolution media feature 504a and sample convolution media feature 504b, and a sample feature pair consisting of sample convolution media feature 504 and sample convolution media feature 504a, that is, the sample convolution media features included in the sample feature pair are adjacent; or, a sample feature pair consisting of two sample convolution media features with an interval among the N sample convolution media features, such as sample convolution media feature 504a and sample convolution media feature 504c, may be formed into a sample feature pair, and no limitation is made here. Wherein, “⊕” 505 refers to feature fusion processing.

[0133] exist Figure 7 The specific acquisition process of the sample convolution media feature 504, the sample convolution media feature 504a, the sample convolution media feature 504b, the sample convolution media feature 504c, the sample feature fusion processing 505, the sample feature fusion processing 505a, and the sample fusion feature 506 can be referred to. Figure 4 For a detailed description of the specific acquisition process of convolutional media feature 401, convolutional media feature 401a, convolutional media feature 401b, convolutional media feature 401c, feature fusion processing 402, convolutional fusion feature 403, and convolutional fusion feature 403a, please refer to Figure 4 The corresponding specific descriptions of the related convolutional media features 401, convolutional media features 401a, convolutional media features 401b, convolutional media features 401c, feature fusion processing 402, convolutional fusion features 403, and convolutional fusion features 403a are not repeated here.

[0134] Step S203: In the initial position recognition model, a first initial channel attention parameter is used to extract features of the sample data to obtain a first type of sample feature corresponding to the sample data feature.

[0135] The specific process of this step can be found in the above Figure 3a The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0136] Step S204: In the initial position recognition model, the second initial channel attention parameter is used to extract the sample data features to obtain the second type of sample features corresponding to the media data features.

[0137] The specific process of this step can be found in the above Figure 3a The specific description of step S103 in the corresponding embodiment will not be repeated here.

[0138] Step S205, in the first type recognition network of the initial position recognition model, feature recognition is performed on the first type of sample features to obtain the position information of the first type of object in the sample media data; in the second type recognition network of the initial position recognition model, feature recognition is performed on the second type of sample features to obtain the position information of the second type of object in the sample media data.

[0139] The specific process of this step can be found in the above Figure 3a The specific description of step S104 in the corresponding embodiment will not be repeated here.

[0140] Please also see Figure 7 ,like Figure 7 As shown, in the first sample identification module 500, assuming that the number r of the first initial channel attention parameters is 2, the first first initial channel attention parameter is used to extract the sample data feature, and the sample attention feature 518 extracted by the first first initial channel attention parameter is obtained. The second first initial channel attention parameter is used to extract the sample data feature, and the sample attention feature 508 extracted by the second first initial channel attention parameter is obtained. At this time, the sample attention feature 508 is determined as a first type of sample feature. The first type of sample feature 508 is feature preprocessed to obtain a first sample probability distribution map 509 and a first sample threshold mapping map 510 corresponding to the first type of object in the sample media data. The first sample probability distribution map 509 and the first sample threshold mapping map 510 are binarized to obtain a first sample approximate binary map 511 corresponding to the first type of object in the sample media data. According to the first sample approximate binary map 511, the position information 512 of the first type of object in the sample media data 503 is determined. The first sample identification module 500 may perform feature identification on the first type sample feature 508 to obtain position information 512 of the first type object in the sample media data.

[0141] See also Figure 7 ,like Figure 7As shown, in the second sample identification module 501, assuming that the number r of the second initial channel attention parameters is 2, the first second initial channel attention parameter is used to extract the sample data feature, and the sample attention feature 519 extracted by the first second initial channel attention parameter is obtained. The second second initial channel attention parameter is used to extract the sample data feature, and the sample attention feature 513 extracted by the second second initial channel attention parameter is obtained. At this time, the sample attention feature 513 is determined as the second type sample feature. The second type sample feature 513 is feature preprocessed to obtain the second sample probability distribution map 514 and the second sample threshold mapping map 515 corresponding to the second type object in the sample media data. The second sample probability distribution map 514 and the second sample threshold mapping map 515 are binarized to obtain the second sample approximate binary map 516 corresponding to the second type object in the sample media data. According to the second sample approximate binary map 516, the position information 517 of the second type object in the sample media data 503 is determined. The second sample identification module 501 may perform feature identification on the second type sample feature 513 to obtain position information 517 of the second type object in the sample media data.

[0142] exist Figure 7 , the specific acquisition process of the first type sample feature 508, the first sample probability distribution map 509, the first sample threshold map 510, the first sample approximate binary map 511, the position information 512 of the first type object in the sample media data, the second type sample feature 513, the second sample probability distribution map 514, the second sample threshold map 515, the second sample approximate binary map 516 and the position information 517 of the second type object in the sample media data can be referred to. Figure 4 For a detailed description, please refer to the specific acquisition process of the first type object feature 405, the first probability distribution map 406, the first threshold mapping map 407, the first approximate binary map 408, the position information 409 of the first type object in the object media data, the second type object feature 411, the second probability distribution map 412, the second threshold mapping map 413, the second approximate binary map 414, and the position information 415 of the second type object in the object media data. Figure 4 The corresponding detailed descriptions of the relevant first type object features 405, the first probability distribution map 406, the first threshold mapping map 407, the first approximate binary map 408, the position information 409 of the first type object in the object media data, the second type object features 411, the second probability distribution map 412, the second threshold mapping map 413, the second approximate binary map 414, and the position information 415 of the second type object in the object media data are not repeated here.

[0143] Step S206, adjusting the parameters of the initial position recognition model according to the position information and the first sample label of the first type of object in the sample media data, and the position information and the second sample label of the second type of object in the sample media data, to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0144] See also Figure 7 ,exist Figure 7 In the method, a first sample loss function is obtained according to the position information of the first type of object in the sample media data and the first sample label. A second sample loss function is obtained according to the position information of the second type of object in the sample media data and the second sample label. A sample equalization loss function is obtained according to the correlation between the position information of the first type of object in the sample media data and the position information of the second type of object in the sample media data; the equalization module 502 is used to perform equalization processing between the first type of object and the second type of object. According to the first sample loss function, the second sample loss function and the sample equalization loss function, the parameters of the initial position recognition model are adjusted to obtain a position recognition model including first channel attention parameters and second channel attention parameters. Through the above training process, the first channel attention parameters and the second channel attention parameters obtained by training have the following characteristics: Figure 4 The effect shown.

[0145] It should be understood that the first sample identification module, the second sample identification module and the sample balancing module can be trained by alternating iterations. The alternating iteration method can include a single module dynamic parameter training method. The single module dynamic parameter training method can be to keep the second sample identification module parameters and the sample balancing module parameters unchanged when training the first sample identification module; the single module dynamic parameter training method can be to keep the first sample identification module parameters and the sample balancing module parameters unchanged when training the second sample identification module; the single module dynamic parameter training method can be to keep the first sample identification module parameters and the first sample identification module parameters unchanged when training the sample balancing module; the trained first sample identification module, the second sample identification module and the sample balancing module can be quickly obtained by maintaining the single module dynamic parameter training method. Optionally, the alternating iteration method can include a dual module dynamic parameter training method. The dual-module dynamic parameter training method can be to train the first sample identification module and the second sample identification module together, and keep the sample balancing module parameters unchanged; the dual-module dynamic parameter training method can be to train the first sample identification module and the sample balancing module together, and keep the second sample identification module parameters unchanged; the dual-module dynamic parameter training method can be to train the second sample identification module and the sample balancing module together, and keep the first sample identification module parameters unchanged; the training between the first sample identification module and the second sample identification module can be trained in an adversarial network manner, the training between the first sample identification module and the sample balancing module can be trained in a weight distribution manner, and the initial power and responsibility distribution can be an equal weight distribution, and the training between the second sample identification module and the sample balancing module can be trained in a weight distribution manner, and the initial power and responsibility distribution can be an equal weight distribution. Through the dual-module dynamic parameter training, a first sample identification module, a second sample identification module, and a sample balancing module with a stronger correlation can be obtained.

[0146] According to the position information of the first type of object in the sample media data and the first sample label, the first sample loss function is obtained, which can be specifically shown in formula ①:

[0147] L1=L S +α×L b +β×L t ①

[0148] In formula ①, L S Represents the loss function of the first probability distribution graph, L t represents the loss function of the first threshold map, L brepresents the loss function of the first approximate binary image, α represents the weight part for the first approximate binary image, for example, the value range of α can be [0.8-1.2], β can represent the weight part for the first threshold mapping image, for example, the value range of β can be [0.8-1.2].

[0149] The loss function of the first probability distribution graph and the loss function of the first approximate binary graph can be binary cross entropy, as shown in formula ②:

[0150]

[0151] In formula ②, S i Can be sample media data, x i It can be the first type of sample feature, y i It may be location information of the first type object in the sample media data.

[0152] The loss function of the first threshold map can be shown in formula ③:

[0153]

[0154] In formula ③, R d Can represent the offset of the annotation threshold, x i * It can be the corresponding feature of the sample data feature in the first threshold map, y i * The first threshold map annotation result may be indicated.

[0155] According to the position information of the second type of object in the sample media data and the second sample label, the second sample loss function is obtained, which can be specifically shown in formula ④:

[0156] L2=L S +α×L b +β×L t (④

[0157] Among them, in formula ④, L S Represents the loss function of the second probability distribution graph, L t represents the loss function of the second threshold map, L b represents the loss function of the second approximate binary image, α represents the weight part for the second approximate binary image, and the value range of α can be [0.8-1.2]. β can represent the weight part for the second threshold mapping image, and the value range of β can be [0.8-1.2].

[0158] The loss function of the second probability distribution graph and the loss function of the second approximate binary graph can be binary cross entropy, as shown in formula ⑤:

[0159]

[0160] Among them, in formula ⑤, S i Can be sample media data, x i It can be the second type of sample feature, y i It may be location information of the second type object in the sample media data.

[0161] According to the correlation between the position information of the first type of object in the sample media data and the position information of the second type of object in the sample media data, a sample balance loss function is obtained, which can be specifically shown in formula ⑥:

[0162] L3=γL1+(1-γ)×L2+L f ⑥

[0163] In formula ⑥, L1 can represent the first sample loss function, L2 represents the second sample loss function, γ can represent the weight of the first sample loss function, (1-γ) can represent the weight of the second sample loss function, L f It can represent the basic balanced loss function when the sample is empty.

[0164] The basic equilibrium loss function can be seen in formula ⑦:

[0165] L f =-α t (1-p t ) δ log(p t ) ⑦

[0166] Among them, in formula ⑦, p t Indicates the probability that the sample media data is the first type of object, α t Represents the equalization coefficient of different samples, α t The value range can be [0.1-0.5], δ represents the mean square order, and the value range of δ can be [1-2].

[0167] See also Figure 8a , Figure 8a is a data structure diagram of a basic balanced loss function provided in an embodiment of the present application. Figure 8a As shown, the horizontal axis of the coordinate graph represents the probability that the sample media data is the first type of object, and the vertical axis of the coordinate graph represents the loss value of the basic balanced loss function. When the value range of the mean square power δ is [1-2], the performance of the basic balanced loss function for simple samples with a predicted probability greater than 0.5 and difficult samples with a probability less than or equal to 0.5 is relatively balanced.

[0168] According to the first sample loss function, the second sample loss function and the sample equalization loss function, the first initial channel attention parameter and the second initial channel attention parameter are adjusted to obtain the first channel attention parameter, the second channel attention parameter and a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0169] The comparison of sample training and test results can be seen in Table 1:

[0170] Table 1

[0171]

[0172]

[0173] Among them, Table 1 can show that both the channel attention convolution module and the equalization module are helpful to improve the separation effect of the first type of objects and the second type of objects. Among them, different from the single convolution module of other deep learning neural networks, the present application adopts two independent channel attention convolution modules, which are used to obtain the media convolution features of printed characters and the media convolution features of printed characters respectively. The first channel attention parameter can be a parameter constructed to obtain printed characters, and the first channel attention parameter has a weight of 95% for obtaining printed characters, and a weight of 5% for obtaining printed characters. The second channel attention parameter can be a parameter constructed to obtain printed characters, and the second channel attention parameter has a weight of 95% for obtaining printed characters, and a weight of 5% for obtaining printed characters. By distinguishing the media data features of different colors, fonts, etc. between printed characters and printed characters, the recall index (Recall) of 4% and the precision index (Precision) of 8% for machine-printed characters and printed characters are improved. This application has strong generalization and can perform character separation processing on different medical bill data. By detecting the text of medical bill data and using the character separation model network, it can well separate the category and position information of printed characters and machine-printed characters, reducing the difficulty of subsequent text recognition of different characters, improving the correctness of identifying the content information related to medical bill data, further reducing the difficulty of extracting structured information of medical bill data content, speeding up the output of structured field information composed of characters, and greatly reducing the time-consuming input of manual review. This application also adds a sample balancing module (FocalLoss) for sample balancing, further improving the character recall index (Recall) of 3% and the precision index (Precision) of 2%.

[0174] The present application is different from the traditional method of adding a layer separation module or other modular methods to the system to perform structured extraction of initial media data in the system for the bill content. The present application replaces the original traditional text detection module and cascades each module to improve the performance of the character separation module used for text detection.

[0175] In addition, this application adopts the feature layer of the public backbone network, which reduces the time consumption of the entire module, so that the network prediction time can be limited to within 500ms. It is used to shorten the network prediction time and can separate the printed characters and printed characters in the bill image in real time.

[0176] See also Figure 7 , where 500 may represent the training part for the first sample loss function, 501 may represent the training part for the second sample loss function, and 502 may represent the training part for the sample equalization loss function.

[0177] In the embodiment of the present application, the first sample loss function, the second sample loss function and the balanced sample loss function are introduced, and position recognition training can be performed according to the first sample loss function, the second sample loss function and the balanced sample loss function. In the actual test, the system of the present application can use the preset character separation model that has been trained in other scenes to fine-tune the character separation model, that is, it can adapt to the application of new scenes, save the pre-preparation time of the character separation system, and more efficiently and conveniently identify the position area of ​​the printed characters and the position area of ​​the printed characters, thereby separating the printed characters from the printed characters. The present application can separate the printed characters from the printed characters on the medical bill data according to different media data characteristics, combined with the color, font and other media data characteristics of the printed characters in the pin-type or template format on the medical bill data, thereby reducing the difficulty of character recognition caused by the overlap and occlusion of the printed characters, thereby reducing the difficulty of structural extraction of the content information of the medical bill data, and improving the complete medical bill data information. With the present application, the different position information of the first type of object and the second type of object can be more accurately identified, and the accuracy of the position information of the separated object media data (such as medical bill data, etc.) can be improved.

[0178] See also Figure 8b , Figure 8b is a flow chart of a method for separating location information provided by an embodiment of the present application. Figure 8bAs shown, the initial media data can be input, and the object media data can be obtained by adjusting the direction. The features of different channels are extracted according to the first channel attention parameter and the second channel attention parameter to obtain the first type object features and the second type object features. The first type object features and the second type object features are input into the balance module for feature balance correction, and the position information of the first type object in the object media data and the position information of the second type object in the object media data can be obtained.

[0179] See also Figure 8c , Figure 8c is a flow chart of a method for separating test location information provided by an embodiment of the present application. Figure 8c As shown, initial media data for testing can be input, and object media data for testing can be obtained by direction adjustment. Feature detection of different channels is performed according to the first channel attention parameter and the second channel attention parameter to obtain first type object features for testing and second type object features for testing. According to the first type object features for testing and the second type object features for testing, position information of the first type object for testing in the object media data and position information of the second type object for testing in the object media data can be obtained.

[0180] For further information, see Figure 9a , Figure 9a : is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Figure 9a As shown, the data processing device 1 may include: an object feature acquisition module 11 , a first feature extraction module 12 , a second feature extraction module 13 and a first feature recognition module 14 .

[0181] The object feature acquisition module 11 is used to acquire the media data features corresponding to the object media data;

[0182] A first feature extraction module 12, configured to extract features of the media data using a first channel attention parameter to obtain first type object features corresponding to the media data features;

[0183] A second feature extraction module 13 is used to extract features of the media data using the second channel attention parameter to obtain second type object features corresponding to the media data features; the first channel attention parameter and the second channel attention parameter are obtained by training under different object types;

[0184] The first feature recognition module 14 is used to perform feature recognition on the first type of object features to obtain the position information of the first type of object in the object media data; input the second type of object features into the position recognition model to perform feature recognition to obtain the position information of the second type of object in the object media data.

[0185] The specific functional implementation of the object feature acquisition module 11, the first feature extraction module 12, the second feature extraction module 13 and the first feature recognition module 14 can be referred to in the above Figure 3a Steps S101 to S104 in the corresponding embodiment will not be described in detail here.

[0186] The data processing device 1 further includes: a semantic analysis module 15 and a direction adjustment module 16 .

[0187] The semantic analysis module 15 is used to obtain the initial media data, perform semantic analysis on the initial media data, and obtain content direction information in the initial media data;

[0188] The direction adjustment module 16 is used to adjust the direction of the initial media data based on the content direction information to obtain the target media data.

[0189] The specific functional implementation of the semantic parsing module 15 and the direction adjustment module 16 can be found in the above Figure 3a The step S101 in the corresponding embodiment will not be described in detail here.

[0190] See also Figure 9a , wherein the semantic parsing module 15 comprises:

[0191] An activation unit 151, configured to perform convolution and activation processing on the initial media data to obtain preliminary activation features corresponding to the initial media data;

[0192] A second conversion unit 152 is used to perform feature conversion processing on the preliminary activation feature based on the hourglass layer to obtain a media adjustment feature;

[0193] The pooling unit 153 is used to perform pooling processing on the media adjustment features to obtain pooled features, and perform full-connection prediction processing on the pooled features to obtain content direction information in the initial media data.

[0194] The specific functional implementation of the activation unit 151, the second conversion unit 152 and the pooling unit 153 can be found in the above Figure 3a The step S101 in the corresponding embodiment will not be described in detail here.

[0195] See also Figure 9a , wherein the object feature acquisition module 11 includes:

[0196] The acquisition unit 111 is used to acquire the object media data, perform feature extraction processing on the object media data, and obtain initial media features of the object media data;

[0197] The first conversion unit 112 is used to perform feature conversion on the initial media features to obtain media data features corresponding to the target media data.

[0198] The specific functional implementation of the acquisition unit 111 and the first conversion unit 112 can be found in the above Figure 3a The step S101 in the corresponding embodiment will not be described in detail here.

[0199] See also Figure 9a , wherein the convolution unit 112 comprises:

[0200] The convolution subunit 1121 is used to perform multi-layer convolution on the initial media features to obtain N convolution media features, where N is a positive integer;

[0201] The upsampling fusion subunit 1122 is used to perform upsampling feature fusion processing on the N convolution media features to obtain convolution fusion features;

[0202] The feature fusion subunit 1123 is used to perform feature fusion processing on the N convolutional media features and the convolutional fusion features to obtain media data features corresponding to the object media data.

[0203] The specific functional implementation of the convolution subunit 1121, the upsampling fusion subunit 1122 and the feature fusion subunit 1123 can be found in the above Figure 3a The step S101 in the corresponding embodiment will not be described in detail here.

[0204] See also Figure 9a , wherein the first feature recognition module 14 further includes:

[0205] A first preprocessing unit 141 is used to perform feature preprocessing on the first type of object features to obtain a first probability distribution map and a first threshold mapping map corresponding to the first type of object in the object media data;

[0206] A first mapping unit 142 is used to perform binarization processing on the first probability distribution map and the first threshold mapping map to obtain a first approximate binary map corresponding to the first type of object in the object media data;

[0207] A first determining unit 143, configured to determine position information of a first type of object in the object media data according to the first approximate binary image;

[0208] A second preprocessing unit 144 is used to perform feature preprocessing on the second type of object features to obtain a second probability distribution map and a second threshold mapping map corresponding to the second type of object in the object media data;

[0209] The second mapping unit 145 is used to perform binarization processing on the second probability distribution map and the second threshold mapping map to obtain a second approximate binary map corresponding to the second type of object in the object media data;

[0210] The second determining unit 146 is configured to determine position information of the second type of object in the object media data according to the second approximate binary image.

[0211] The specific functional implementation of the first preprocessing unit 141, the first mapping unit 142, the first determining unit 143, the second preprocessing unit 144, the second mapping unit 145 and the second determining unit 146 can be referred to above. Figure 3a The step S104 in the corresponding embodiment will not be described in detail here.

[0212] In the embodiment of the present application, a first sample loss function, a second sample loss function and a balanced sample loss function are introduced, and position recognition training can be performed based on the first sample loss function, the second sample loss function and the balanced sample loss function. The present application can more accurately identify the different position information of the first type of object and the second type of object, and improve the accuracy of the position information of the separated object media data (such as medical bill data, etc.). The present application can effectively learn the concentrated and stably distributed printed character media data features and printed character media data features for medical bill data in different regions. During actual testing, the system of the present application can use a preset character separation model that has been trained for other scenarios, and fine-tune the character separation model, that is, it can adapt to the application of new scenarios, save the initial preparation time of the character separation system, and more efficiently and conveniently identify the position area of ​​printed characters and the position area of ​​printed characters, and then separate printed characters from printed characters. For further information, please refer to Figure 9b , Figure 9b : is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Figure 9b As shown, the data processing device 2 may include: a sample label acquisition module 21, a sample feature acquisition module 22, a third feature extraction module 23, a fourth feature extraction module 24, a second feature recognition module 25 and a model determination module 26.

[0213] A sample label acquisition module 21 is used to acquire a first sample label corresponding to a first type of object in sample media data, and acquire a second sample label corresponding to a second type of object in sample media data;

[0214] The sample feature acquisition module 22 is used to input the sample media data into the initial position recognition model, and obtain the sample data features corresponding to the sample media data in the initial position recognition model;

[0215] A third feature extraction module 23 is used to extract features of the sample data using the first initial channel attention parameter in the initial position recognition model to obtain a first type of sample feature corresponding to the sample data feature;

[0216] A fourth feature extraction module 24 is used to extract features of the sample data using the second initial channel attention parameter in the initial position recognition model to obtain a second type of sample feature corresponding to the media data feature;

[0217] The second feature recognition module 25 is used to perform feature recognition on the first type of sample features in the first type recognition network of the initial position recognition model to obtain the position information of the first type of object in the sample media data; perform feature recognition on the second type of sample features in the second type recognition network of the initial position recognition model to obtain the position information of the second type of object in the sample media data;

[0218] The model determination module 26 is used to adjust the parameters of the initial position recognition model according to the position information of the first type of object in the sample media data and the first sample label, and the position information of the second type of object in the sample media data and the second sample label, to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0219] The specific functional implementation of the sample label acquisition module 21, the sample feature acquisition module 22, the third feature extraction module 23, the fourth feature extraction module 24, the second feature recognition module 25 and the model determination module 26 can be found in the above Figure 5 Steps S201 to S206 in the corresponding embodiment are not described in detail here.

[0220] See also Figure 9b , wherein the model determination module 26 comprises:

[0221] A first function acquisition unit 261 is used to obtain a first sample loss function according to the position information of the first type of object in the sample media data and the first sample label;

[0222] A second function acquisition unit 262, configured to obtain a second sample loss function according to the position information of the second type of object in the sample media data and the second sample label;

[0223] The equalization function acquisition unit 263 is used to obtain a sample equalization loss function according to the correlation between the position information of the first type of object in the sample media data and the position information of the second type of object in the sample media data;

[0224] The model determination unit 264 is used to adjust the parameters of the initial position recognition model according to the first sample loss function, the second sample loss function and the sample equalization loss function to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0225] The specific functional implementation of the first function acquisition unit 261, the second function acquisition unit 262, the balance function acquisition unit 263 and the model determination unit 264 can be referred to above. Figure 5 The step S206 in the corresponding embodiment will not be described in detail here.

[0226] In an embodiment of the present application, a first sample loss function, a second sample loss function and a balanced sample loss function are introduced, and position recognition training can be performed according to the first sample loss function, the second sample loss function and the balanced sample loss function. By adopting the present application, different position information of the first type of object and the second type of object can be more accurately identified, and the accuracy of the position information of the separated object media data (such as medical bill data, etc.) can be improved. The present application can separate the printed characters on the medical bill data from the printed characters according to different media data characteristics, combined with the color, font and other media data characteristics of the printed characters in the pin-type or template format on the medical bill data and the printed characters, thereby reducing the difficulty of character recognition caused by the overlap and occlusion of the printed characters, thereby reducing the difficulty of structured extraction of the content information of the medical bill data, and improving the complete medical bill data information.

[0227] For further information, see Fig.10 , Fig.10 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.10As shown, the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. Fig.10 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0228] exist Fig.10 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0229] Acquire media data features corresponding to object media data; use first channel attention parameters to perform feature extraction on media data features to obtain first type object features corresponding to media data features; use second channel attention parameters to perform feature extraction on media data features to obtain second type object features corresponding to media data features; the first channel attention parameters and the second channel attention parameters are obtained by training under different object types; perform feature recognition on first type object features to obtain position information of first type objects in object media data; perform feature recognition on second type object features to obtain position information of second type objects in object media data.

[0230] In one embodiment, the processor 1001 further performs the following steps:

[0231] Initial media data is obtained, and semantic analysis is performed on the initial media data to obtain content direction information in the initial media data; and the direction of the initial media data is adjusted based on the content direction information to obtain object media data.

[0232] In one embodiment, when the processor 1001 performs semantic parsing on the initial media data to obtain the content direction information in the initial media data, the processor 1001 further performs the following steps:

[0233] The initial media data is convolved and activated to obtain initial activation features corresponding to the initial media data; the initial activation features are converted based on the hourglass layer to obtain media adjustment features; the media adjustment features are pooled to obtain pooled features, and the pooled features are fully connected and predicted to obtain content direction information in the initial media data.

[0234] In one embodiment, when acquiring the media data features corresponding to the object media data, the processor 1001 further performs the following steps:

[0235] Obtain object media data, perform feature extraction processing on the object media data, and obtain initial media features of the object media data; perform feature conversion on the initial media features to obtain media data features corresponding to the object media data.

[0236] In one embodiment, when the processor 1001 performs feature conversion on the initial media feature to obtain the media data feature corresponding to the target media data, the processor 1001 further performs the following steps:

[0237] Perform multi-layer convolution on the initial media features to obtain N convolution media features, where N is a positive integer; perform upsampling feature fusion processing on the N convolution media features to obtain convolution fusion features; perform feature fusion processing on the N convolution media features and the convolution fusion features to obtain media data features corresponding to the object media data.

[0238] In one embodiment, when the processor 1001 performs feature recognition on the first type of object features to obtain the position information of the first type of object in the object media data, and performs feature recognition on the second type of object features to obtain the position information of the second type of object in the object media data, the processor 1001 further performs the following steps:

[0239] Perform feature preprocessing on features of first type objects to obtain a first probability distribution map and a first threshold mapping map corresponding to the first type objects in the object media data; perform binarization processing on the first probability distribution map and the first threshold mapping map to obtain a first approximate binary map corresponding to the first type objects in the object media data; determine the position information of the first type objects in the object media data based on the first approximate binary map; perform feature preprocessing on features of second type objects to obtain a second probability distribution map and a second threshold mapping map corresponding to the second type objects in the object media data; perform binarization processing on the second probability distribution map and the second threshold mapping map to obtain a second approximate binary map corresponding to the second type objects in the object media data; determine the position information of the second type objects in the object media data based on the second approximate binary map.

[0240] In one embodiment, the processor 1001 obtains a first sample label corresponding to a first type of object in sample media data, and obtains a second sample label corresponding to a second type of object in the sample media data; inputs the sample media data into an initial position recognition model, and obtains sample data features corresponding to the sample media data in the initial position recognition model; in the initial position recognition model, a first initial channel attention parameter is used to perform feature extraction on the sample data features to obtain first type sample features corresponding to the sample data features; in the initial position recognition model, a second initial channel attention parameter is used to perform feature extraction on the sample data features to obtain second type sample features corresponding to the media data features; in a first type recognition network of the initial position recognition model, feature recognition is performed on the first type sample features to obtain position information of the first type of object in the sample media data; in a second type recognition network of the initial position recognition model, feature recognition is performed on the second type sample features to obtain position information of the second type of object in the sample media data; according to the position information and the first sample label of the first type of object in the sample media data, and the position information and the second sample label of the second type of object in the sample media data, the parameters of the initial position recognition model are adjusted to obtain a position recognition model including the first channel attention parameter and the second channel attention parameter.

[0241] In one embodiment, when the processor 1001 adjusts the parameters of the initial position recognition model according to the position information of the first type of object in the sample media data and the first sample label, and the position information of the second type of object in the sample media data and the second sample label to obtain the position recognition model including the first channel attention parameter and the second channel attention parameter, the processor 1001 further performs the following steps:

[0242] A first sample loss function is obtained according to position information of a first type of object in sample media data and a first sample label; a second sample loss function is obtained according to position information of a second type of object in sample media data and a second sample label; a sample equalization loss function is obtained according to the correlation between position information of the first type of object in sample media data and position information of the second type of object in sample media data; and parameters of an initial position recognition model are adjusted according to the first sample loss function, the second sample loss function and the sample equalization loss function to obtain a position recognition model including first channel attention parameters and second channel attention parameters.

[0243] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 2 , Figure 3a , Figure 3b , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8a , Figure 8b as well as Figure 8c The description of the data processing method in the corresponding embodiment can also be performed as described above. Figure 9a The description of the data processing device 1 in the corresponding embodiment can also be performed as described above. Figure 9b The description of the data processing device 2 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0244] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, Figure 2 , Figure 3a , Figure 3b , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8a , Figure 8b as well as Figure 8c For details on the data processing methods provided in each step, please refer to the above Figure 2 , Figure 3a , Figure 3b , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8a , Figure 8b as well as Figure 8c The implementation methods provided by each step will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail here either.

[0245] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0246] The embodiment of the present application also provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device can execute the above Figure 2 , Figure 3a , Figure 3b , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8a , Figure 8b as well as Figure 8c The description of the data processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here.

[0247] The term "comprising" and any variations thereof in the description, claims and drawings of the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, devices, products or equipment.

[0248] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0249] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0250] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Acquire media data features corresponding to the object media data through the position recognition model; the media data features include media channel features under k channels; k is a positive integer; the k channels include channels strongly associated with the first type of object and channels strongly associated with the second type of object, and the channels strongly associated with the first type of object and the channels strongly associated with the second type of object are not completely the same; In the first recognition module, the first channel weights for the k media channel features in the first channel attention parameter are used to extract features of the k media channel features to obtain first type object features corresponding to the media data features; In the second recognition module, the second channel weights of the k media channel features in the second channel attention parameters are used to extract features of the k media channel features to obtain second type object features corresponding to the media data features; the first channel attention parameters and the second channel attention parameters are obtained by training under different object types; Performing feature recognition on the first type of object features by the first recognition module to obtain location information of the first type of object in the object media data; The second identification module performs feature recognition on the features of the second type of objects to obtain location information of the second type of objects in the object media data.

2. The method according to claim 1, characterized in that The method further comprises: Acquire initial media data, perform semantic analysis on the initial media data, and obtain content direction information in the initial media data; The initial media data is directionally adjusted based on the content direction information to obtain the object media data.

3. The method according to claim 2, characterized in that The performing semantic parsing on the initial media data to obtain content direction information in the initial media data includes: Performing convolution and activation processing on the initial media data to obtain preliminary activation features corresponding to the initial media data; Performing feature conversion processing on the preliminary activation feature based on the hourglass layer to obtain a media adjustment feature; The media adjustment features are pooled to obtain pooled features, and the pooled features are fully connected predicted to obtain content direction information in the initial media data.

4. The method according to claim 1, characterized in that The acquiring of the media data features corresponding to the object media data by using the location recognition model includes: Acquire object media data, and perform feature extraction processing on the object media data through a position recognition model to obtain initial media features of the object media data; The initial media features are subjected to feature conversion to obtain media data features corresponding to the target media data.

5. The method according to claim 4, characterized in that The performing feature conversion on the initial media feature to obtain the media data feature corresponding to the object media data includes: Perform multi-layer convolution on the initial media features to obtain N convolution media features, where N is a positive integer; Performing upsampling feature fusion processing on the N convolutional media features to obtain convolutional fusion features; Feature fusion processing is performed on the N convolutional media features and the convolutional fusion features to obtain media data features corresponding to the object media data.

6. The method according to claim 1, characterized in that The performing feature recognition on the first type of object features to obtain location information of the first type of object in the object media data, and performing feature recognition on the second type of object features to obtain location information of the second type of object in the object media data, includes: Performing feature preprocessing on the first type of object features to obtain a first probability distribution map and a first threshold mapping map corresponding to the first type of object in the object media data; Binarizing the first probability distribution map and the first threshold map to obtain a first approximate binary map corresponding to the first type of object in the object media data; Determine, according to the first approximate binary image, position information of the first type of object in the object media data; Performing feature preprocessing on the second type of object features to obtain a second probability distribution map and a second threshold mapping map corresponding to the second type of object in the object media data; Binarizing the second probability distribution map and the second threshold map to obtain a second approximate binary map corresponding to the second type of object in the object media data; According to the second approximate binary image, position information of the second type object in the object media data is determined.

7. A data processing method, characterized in that: include: Obtaining a first sample tag corresponding to a first type of object in the sample media data, and obtaining a second sample tag corresponding to a second type of object in the sample media data; Inputting the sample media data into an initial position recognition model, and obtaining sample data features corresponding to the sample media data in the initial position recognition model; the sample data features include channel features under k channels, where k is a positive integer; the k channels include channels strongly associated with the first type of object and channels strongly associated with the second type of object, and the channels strongly associated with the first type of object and the channels strongly associated with the second type of object are not completely the same; In the initial position recognition model, a first channel weight for the channel features under the k channels in the first initial channel attention parameter is used to extract the channel features under the k channels, so as to obtain a first type of sample feature corresponding to the sample media data feature; In the initial position recognition model, the second channel weights of the channel features under the k channels in the second initial channel attention parameters are used to extract the channel features under the k channels to obtain the second type of sample features corresponding to the sample media data features; In the first type recognition network of the initial position recognition model, feature recognition is performed on the first type sample features to obtain position information of the first type object in the sample media data; in the second type recognition network of the initial position recognition model, feature recognition is performed on the second type sample features to obtain position information of the second type object in the sample media data; According to the position information of the first type of object in the sample media data and the first sample label, as well as the position information of the second type of object in the sample media data and the second sample label, the parameters of the initial position recognition model are adjusted to obtain a position recognition model including first channel attention parameters and second channel attention parameters.

8. The method according to claim 7, characterized in that The method further comprises: adjusting parameters of the initial position recognition model according to the position information of the first type of object in the sample media data and the first sample label, and the position information of the second type of object in the sample media data and the second sample label to obtain a position recognition model including a first channel attention parameter and a second channel attention parameter, including: Obtaining a first sample loss function according to the position information of the first type of object in the sample media data and the first sample label; Obtaining a second sample loss function according to the position information of the second type of object in the sample media data and the second sample label; Obtaining a sample equalization loss function according to the correlation between the position information of the first type of object in the sample media data and the position information of the second type of object in the sample media data; According to the first sample loss function, the second sample loss function and the sample equalization loss function, the parameters of the initial position recognition model are adjusted to obtain a position recognition model including first channel attention parameters and second channel attention parameters.

9. A data processing device, characterized in that: include: An object feature acquisition module, used to acquire media data features corresponding to the object media data through a position recognition model; the media data features include media channel features under k channels; k is a positive integer; the k channels include channels strongly associated with the first type of object and channels strongly associated with the second type of object, and the channels strongly associated with the first type of object and the channels strongly associated with the second type of object are not completely the same; A first feature extraction module is used to extract features of the k media channel features using the first channel weights for the k media channel features in the first channel attention parameter in the first recognition module to obtain first type object features corresponding to the media data features; A second feature extraction module is used to extract features of the k media channel features using the second channel weights for the k media channel features in the second channel attention parameters in the second recognition module to obtain second type object features corresponding to the media data features; the first channel attention parameters and the second channel attention parameters are obtained by training under different object types; The first feature recognition module is used to perform feature recognition on the first type of object features through the first recognition module to obtain the position information of the first type of object in the object media data; and input the second type of object features into the position recognition model through the second recognition module to perform feature recognition to obtain the position information of the second type of object in the object media data.

10. A data processing device, characterized in that: include: A sample label acquisition module, used to acquire a first sample label corresponding to a first type of object in sample media data, and acquire a second sample label corresponding to a second type of object in the sample media data; A sample feature acquisition module, used to input the sample media data into an initial position recognition model, and obtain sample data features corresponding to the sample media data in the initial position recognition model; the sample data features include channel features under k channels, k is a positive integer; the k channels include channels strongly associated with the first type of object and channels strongly associated with the second type of object, and the channels strongly associated with the first type of object and the channels strongly associated with the second type of object are not completely the same; A third feature extraction module is used to extract features of the channel features under the k channels using the first channel weights of the channel features under the k channels in the first initial channel attention parameter in the initial position recognition model to obtain a first type of sample feature corresponding to the sample media data feature; a fourth feature extraction module, configured to extract features of the channel features under the k channels by using the second channel weights of the channel features under the k channels in the second initial channel attention parameters in the initial position recognition model, so as to obtain second type sample features corresponding to the sample media data features; A second feature recognition module is used to perform feature recognition on the first type of sample features in the first type recognition network of the initial position recognition model to obtain position information of the first type of object in the sample media data; and perform feature recognition on the second type of sample features in the second type recognition network of the initial position recognition model to obtain position information of the second type of object in the sample media data; A model determination module is used to adjust the parameters of the initial position recognition model according to the position information of the first type of object in the sample media data and the first sample label, as well as the position information of the second type of object in the sample media data and the second sample label, to obtain a position recognition model including first channel attention parameters and second channel attention parameters.

11. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store program code, and the processor is used to call the program code to execute the method described in any one of claims 1-6 or the method described in any one of claims 7-8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 8.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 or the steps of the method according to any one of claims 7 to 8 are implemented.

Images